Generic transcoding service for client-server communication

ABSTRACT

In particular embodiments, a computing device may receive device data from a client device. The computing device may encode the data for transmission to a remote desktop client. The re-encoding may be performed according to a pre-defined import/export protocol.

FIELD

The disclosure relates in general to client-server communication, andmore particularly to, a system and method for facilitating client-servercommunication.

BACKGROUND

Many known client-side protocols (CSP) for communicating with a remotecomputing device, for example, remote desktop protocol (RDP),independent computing architecture (ICA), personal computer overInternet protocol (PCoIP), and virtual network computing (VNC), areknown. Under these protocols, a CSP's back-end (CBE) may becommunicatively coupled with a server and the client may run anapplication within an operating system of the remote computing devicevia the server. The data from the application within the operatingsystem of the remote computing device may be presented via the CSP'sfront-end (CFE) input/output devices (e.g., screen, speaker, etc.) ofthe client directly or by alternate protocol, and similarly, input maybe received via the input devices (e.g., keyboard, mouse, microphone,etc.) or by alternate protocol of the user-input-client (UIC). However,other approaches for specialized proxy interfacing with the CBE or CFEdata perspective may be extremely advantageous and desirable.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of a client-server computer system with aservice compatible client.

FIG. 2 illustrates an example of a client-server computer system with aservice incompatible client in accordance with one aspect of thedisclosure.

FIG. 3 illustrates an example of a proxy machine in accordance with oneaspect of the disclosure.

FIG. 4 illustrates an example of a translation module in accordance withone aspect of the disclosure.

FIG. 5 is a conceptual block diagram illustrating an example of acomputing device.

FIG. 6 illustrates a simplified diagram of an example of a networksystem.

FIG. 7 illustrates an example of a process of facilitating creating atunnel and channels between the proxy machine and the remote servercomputing device in accordance with one aspect of the disclosure.

FIG. 8 illustrates an example of a process of facilitating communicationbetween the service incompatible client and the remote server computingdevice in accordance with one aspect of the disclosure.

FIG. 9 illustrates an example of a client-server computer system inaccordance with one aspect of the disclosure.

FIG. 10 illustrates an example of a process of translating a messagefrom a format associated with a sending computing device to a formatassociated with the receiving computing device in accordance with oneaspect of the disclosure.

FIG. 11 illustrates an example of a process of authenticating a messagereceived in a format associated with a client computing device format toa format associated with a server computing device in accordance withone aspect of the disclosure.

FIG. 12 illustrates an example of a process of authenticating a messagereceived in a format associated with the server computing device to aformat associated with a client computing device in accordance with oneaspect of the disclosure.

FIG. 13 is a block diagram illustrating an example of a client-servercommunication system.

FIG. 14 is a block diagram illustrating an example of a client-servercommunication system configured to transcode data between remote desktopprotocol and hypertext transfer protocol.

FIG. 15A is a flow chart illustrating an example of an operation of asetup engine.

FIG. 15B is an example of a device including instructions for anoperation of a setup engine.

FIG. 15C is an example of a device including one or more modules for anoperation of a setup engine.

FIG. 16A is a flow chart illustrating an example of an operation of aconnection engine.

FIG. 16B is an example of a device including instructions for anoperation of a connection engine.

FIG. 16C is an example of a device including one or more modules for anoperation of a connection engine.

FIG. 17 is a conceptual block diagram of a local device virtualizationsystem, in accordance with various aspects of the subject technology.

FIG. 18 illustrates an example connection engine.

FIG. 19 illustrates CSP connections through Front-end and Back-endinterfaces.

FIG. 20 illustrates a Runtime API framework.

DETAILED DESCRIPTION

The detailed description set forth below is intended as a description ofvarious configurations of the subject technology and is not intended torepresent the only configurations in which the subject technology may bepracticed. The appended drawings are incorporated herein and constitutea part of the detailed description. The detailed description includesspecific details for the purpose of providing a thorough understandingof the subject technology. However, it will be apparent to those skilledin the art that the subject technology may be practiced without thesespecific details. In some instances, well-known structures andcomponents are shown in block diagram form in order to avoid obscuringthe concepts of the subject technology. Like components are labeled withidentical element numbers for ease of understanding.

In one aspect, the subject technology relates to a CBE remote procedurecall frame and filter (RPCFF) host that facilitates interfacing aproprietary remote server (e.g., a desktop computer running a MicrosoftWindow® operating system implementing Microsoft Remote Procedure Call®)with a non-proprietary client computing device (e.g., a laptop computerimplementing a Linux Ubuntu® operating system). The RPCFF host mayfacilitate receiving, from the client computing device not configured toimplement the proprietary remote procedure call protocol (e.g.,Microsoft RPC), a request to communicate with the remote serverconfigured to implement the proprietary remote procedure call protocol.The RPCFF host may facilitate providing, via a remote procedure calllayer, network level authentication for the client computing device. TheRPCFF host may provide a data path for a logical role on the remoteserver without implementing a server stack in the proprietary remoteprocedure call protocol. The logical role may be configured to implementthe proprietary remote procedure call protocol. The RPCFF host maydefine a communication syntax, for the data path, between the clientcomputing device and the logical role. The communication syntax may bedifferent from the proprietary remote procedure call protocol. Thecommunication syntax may involve translating between a universalcommunication protocol (e.g. Secure Shell) and the proprietary remoteprocedure call protocol. The RPCFF host may facilitate communicatingdata, according to the communication syntax, via the data path, betweenthe client computing device and the logical role.

In one implementation, the subject technology relates to a remotegateway client proxy. The CBE proxy assists a service incompatibleclient computing device in communicating with a remote server computingdevice and machines in a service, including a network-based procedurecall interface and a gateway interface.

Client-server computer systems are oftentimes implemented to allow usersof client computing devices to access data stored on server computingdevices that are located remote to the client computing devices.However, one disadvantage of client-server computer systems is that aparticular client computing device might not be able to interface with aparticular server computing device if the client computing device lacksan operating system that is associated with the server computing device.For example, a server computing device implementing a specializedoperating system, such as Microsoft Windows Server 2008, may only beable to interact with client computing devices implementing certainMicrosoft operating systems, such as Windows XP or Vista. Clientcomputing devices implementing other operating systems, such as GoogleAndroid or Apple IOS operating systems, may not be able to fullyinteract with the server computing device. As the foregoing illustrates,what is needed in the art is a technique to allow a client computingdevice implementing an arbitrary operating system to fully interact witha server computing device implementing a specialized operating system.

Disadvantages of known client-server computing systems include thatthese systems rely heavily on proprietary protocols associated with theserver computing device. For example, remote procedure call (RPC) is aMicrosoft proprietary protocol. Only operating systems provisioned byMicrosoft with RPC can use any Microsoft role, such as terminal servicesgateway (TSGW). This requires a very large footprint of code. Some ofthe shortcomings are that only Microsoft supplies the solution, the codesize is large, and access through this mechanism is by design limited toMicrosoft products.

In one aspect, the subject technology provides, among other things, aclient-side solution that can interface Microsoft's Remote DesktopGateway/Terminal Services Gateway (RDGW/TSGW) service, which impliesthat Microsoft's RPC are used. Therefore, in theory, it is not possibleto interface a non-Microsoft client to Microsoft's TSGW since thatnon-Microsoft client doesn't have Microsoft's RPC capabilities. It istherefore the intrinsic nature of one aspect of the subject technologyto “bridge” the RPC requirements without actually having an RPC softwarestack. This is accomplished by “hard coding” all input and outputpackets with data structures. Only data payloads change. Non-payloadinformation remains constant. This is referred to in the industry as a“specialized packet filter.”

In one approach, RPC is utilized to implement TSGW/RDGW service access.One alternative described herein would involve a specialized packetfilter, or a Microsoft RPC clone.

In one aspect, advantages and benefits of the subject technology mayinclude interfacing of RPC for tunneling of data through a firewall in aMicrosoft environment with a non-Microsoft client. Furthermore, theability to tunnel data by use of RPC without an RPC stack using aspecialized packet filter is another objective. As a result, aspects ofthe subject technology can provide a clear competitive advantage thatenables clients to access within corporate firewalls using standardMicrosoft services or roles, or services or roles that traditionallyhave required the client computing device to have an operating systemthat is “compatible” with the server computing device.

First Example of Client-Server Computer System

FIG. 1 illustrates an example of a client-server computer system 100. Asshown, the computer system 100 includes service compatible client 110having CBE access to firewall 120, service 125, and remote servercomputing device 160. The service 125 includes binding interface 130,network-based procedure call interface 140, and gateway interface 150.

In one aspect, the service 125 is a Microsoft service, the network-basedprocedure call interface 140 is a remote procedure call (RPC) server,and the gateway interface 150 is a terminal services gateway (TSGW) orremote desktop gateway (RDGW) server. The service compatible client 110may implement a Microsoft Windows operating system, such as XP or Vista.

The service compatible client 110 can be a laptop computer, a desktopcomputer, a tablet computer, a personal digital assistant (PDA), amobile phone, or any similar device. The service compatible client 110may implement an operating system compatible with service 125. As shown,the service compatible client 110 is connected with the bindinginterface 130 over the firewall 120, which may function to create ordeny network connections based on a set of rules. In one aspect, thecommunication between the service compatible client 110 and the service125 is implemented in RPC.

As shown, the binding interface 130 includes a memory 132. The memory132 includes a binding in module 134 and a binding out module 136. Thebinding in module 134 and binding out module 136 may be implemented inall software, all hardware or a combination of software and hardware. Asillustrated, the binding in module 134 receives input messages from theservice compatible client 110 in a format associated with thenetwork-based procedure call communications of the service 125 (e.g.,RPC if the service 125 implements Microsoft Server) and transmits theseinput messages to the network-based procedure call interface 140. Thebinding out module 136 receives messages from the network-basedprocedure call interface 140 and provides these messages to the servicecompatible client 110. In one aspect, the service 125 is a Microsoftservice, and the messages passing through the binding in module 134 andthe binding out module 136 are in RPC.

In one aspect, the network-based procedure call interface 140 is an RPCserver that implements an RPC protocol to allow commands in messagesfrom the service compatible client 110 to be executed on the remoteserver computing device 160. In one aspect, the commands in the messagesfrom the service compatible client 110 are bound to an address space ofthe remote server computing device 160 using the gateway interface 150.In one aspect, the gateway interface 150 implements Microsoft terminalservices gateway (TSGW), Microsoft remote desktop gateway (RDGW) or asimilar protocol that is associated with the operating system of theserver computing device 160.

In one aspect, a tunnel 170 is created between the service compatibleclient 110 and the remote server computing device 160, facilitatingcommunication between the service compatible client 110 and remoteserver computing device 160. In one aspect the tunnel 170 includes achannel 175 connecting the service compatible client 110 with the remoteserver computing device 160. In one aspect, there are multiple remoteserver computing devices 160 connected to a single service compatibleclient 110 using a single tunnel 170 and multiple channels 175.

Second Example of Client-Server Computer System

FIG. 2 illustrates an example of a client-server computer system 200with a service incompatible client 210 in accordance with one aspect ofthe disclosure. As shown, the computer system 200 includes serviceincompatible client 210, proxy machine 220 which may include CSP,firewall 120, service 125, and remote server computing device 160. Theservice 125 includes binding interface 130, network based procedure callinterface 140, and gateway interface 150.

When provided as complete service, firewall 120 may be located beforeProxy Machine 220. FIG. 19 illustrates an example of CSP connectionsthrough front-end and back-end interfaces. A firewall may exist betweenthe CSP front-end and the client protocol. A firewall may also existbetween the CSP back-end and a CSP-compatible server.

In one aspect, the service 125 is a Microsoft service, the network-basedprocedure call interface 140 is a remote procedure call (RPC) server,and the gateway interface 150 is a terminal services gateway (TSGW) orremote desktop gateway (RDGW) server. The service incompatible client210 may implement a non-Microsoft operating system, such as Apple iOS orGoogle Android.

The gateway interface 150 may provide a role service that allowsauthorized remote client computing devices to connect to networkresources on an internal corporate or private network. The networkresources can be servers or computers with a remote desktop protocol(e.g., Microsoft Remote Desktop Protocol®) enabled.

The gateway interface 150 may use Remote Desktop Protocol (RDP) overHTTPS to establish a secure, encrypted connection between the remoteclient computing devices and the internal network resources which theremote client computing devices attempt to access.

The gateway interface 150 may provide a client computing device withaccess to all network resources. Alternatively, the gateway interface150 may provide a client computing device with access to only one ormore network resources that the client computing device attempts toaccess, for example, via a point-to-point RDP connection.

In one aspect, a remote procedure call may be an inter-processcommunication that allows a computer program to cause a subroutine orprocedure to execute in another address space (commonly on anothercomputer on a shared network) without the programmer explicitly codingthe details for this remote interaction. For example, a client computingdevice may cause a subroutine or procedure to execute in the addressspace of the server. In one implementation, the programmer may writeessentially the same code whether the subroutine is local to theexecuting program, or remote. In one example, remote procedure call mayinclude remote invocation or remote method invocation.

In one example implementation, a remote procedure call may be initiatedby the client, which may send a request message to a known remote serverto execute a specified procedure with supplied parameters. The remoteserver may send a response to the client, and the application continuesits process. In one example, the server may receive one remote procedurecall at a time from the client. Alternatively, the client may send oneor more asynchronous requests to the server, for example, one or moreXHTTP calls. (4) The local operating system on the server may pass theincoming packets to the server

One difference between remote procedure calls and local calls is thatremote procedure calls may, in some cases, fail due to unpredictablenetwork problems. In some implementations, callers of remote procedurecalls may deal with such failures without knowing whether the remoteprocedure was actually invoked. In some example, remote procedure callsmay be used in carefully written low-level subsystems.

In one aspect, a sequence of events during a remote procedure call maybe as follows. However, other sequences of events during a remoteprocedure call may also be used in conjunction with the subjecttechnology.

(1) The client may call the client stub. The call may be a localprocedure call. Parameters may be pushed on to an OSI stack.

(2) The client stub may pack the parameters (e.g., using marshalling)into a message and make a system call to send the message.

(3) The client's local operating system may send the message from theclient to the server. stub.

(5) Finally, the server stub may call the server procedure. The replymay follow, for example, the same steps in the reverse direction.

In one example, a stub may be a piece of code used for convertingparameters passed during a remote procedure call.

A remote procedure call may, in one example, allow a client to remotelycall procedures on a server. The client and server may use differentaddress spaces, so conversion of parameters used in a function call mayhave to be performed to allow the values of parameters passed during theremote procedure call to be used, as pointers to the memory may point todifferent data on the client and on the server. The client and servermay also use different data representations even for simple parameters(e.g., big-endian versus little-endian for integers). Stubs are used toperform the conversion of the parameters, so a remote procedure callfrom the client looks, to the server, like a local procedure call forthe server, and vice versa.

In one implementation, stub libraries are installed on client and serverside. A client stub is responsible for conversion of parameters used ina function call and deconversion of results passed from the server afterexecution of the function. A server skeleton, the stub on server side,is responsible for deconversion of parameters passed by the client andconversion of the results after the execution of the function.

A stub may be generated either manually or automatically.

Manually: In this approach, the remote procedure call implementer mayprovide a set of translation functions from which a programmer canconstruct stubs.

Automatically: An interface description language (IDL), may be used fordefining the interface between client and server. For example, aninterface definition has information to indicate whether, each argumentis input, output or both—only input arguments may need to be copied fromclient to server and only output elements may need to be copied fromserver to client.

In one example, in a remote procedure call, the client and the serverare connected via a network (e.g., the Internet, an intranet or aVirtual Private Network), via a remote desktop protocol connection(e.g., via Microsoft Remote Desktop Protocol®, via Citrix IndependentComputing Architecture (ICA)®, or via VMWare VMView®), via a wiredconnection, or via a wireless connection. In one example, anetwork-based procedure call may be a remote procedure call transmittedfrom a client to a server via a network (e.g., the Internet). The clientand the server may be connected via a network.

The service incompatible client 210 is coupled with a proxy machine 220,which communicates with the binding interface 130. The binding interface130 may function to bind a proprietary (e.g., Microsoft®) clientinterface to the proprietary (e.g., Microsoft®) server interface. Forexample, the binding interface 130 may bind the Windows® clientinterface to a Microsoft® RPC interface. In a first aspect, the proxymachine 220 is a component of the service incompatible client 210. In asecond aspect, the proxy machine 220 may be located outside the serviceincompatible client 210 on the same side of the firewall 120 as theservice incompatible client 210, as illustrated. In a third aspect, theproxy machine 220 is located within the firewall 120. In a fourthaspect, the proxy machine 220 is a component of the service 125. In afifth aspect, the proxy machine 220 is located on the same side of thefirewall as the service 125 but is not a component of the service 125.Persons skilled in the art may realize that other locations of the proxymachine are possible and consistent with the subject technology.

The service incompatible client 210 can be a laptop computer, a desktopcomputer, a tablet computer, a personal digital assistant (PDA), amobile phone, or any similar device. The service incompatible client 210may implement an arbitrary operating system incompatible with service125. For example, if service 125 implements Microsoft Server, theservice incompatible client 210 may implement an Apple IOS or a GoogleAndroid operating system.

As shown, the service incompatible client 210 is connected with theproxy machine 220. In one aspect, the proxy machine 220 is internal tothe service incompatible client 210 and the connection between proxymachine 220 and service incompatible client 210 is a hard wire or otherinternal connection. In another aspect, the proxy machine 220 isexternal to the service incompatible client 210 and a native Internetprotocol (IP) connection is implemented between the service incompatibleclient 210 and the proxy machine 220. In one aspect, the serviceincompatible client 210 can only communicate using native IPcommunications and the proxy machine 220 handles the communications withthe service 125 transparently to the service incompatible client 210.The proxy machine 220 is also connected with the binding interface 130.In one aspect, the service incompatible client 210 communicates with theproxy machine using a native IP communication technique or a similaruniversal format. However, the service 125 is designed to acceptmessages in the network-based procedure call communication format forthe service 125 (e.g., RPC if the service 125 is running MicrosoftServer). The proxy machine 220 converts messages from the universalformat (e.g., native IP communication) to the network-based procedurecall format of the service 125 (e.g., RPC) based on an encoding for thenetwork-based procedure call format of the service 125. The operation ofthe proxy machine 220 is described in more detail in conjunction withFIGS. 3-4 below.

As shown, the binding interface 130 includes a memory 132. The memory132 includes a binding in module 134 and a binding out module 136. Thebinding in module 134 and binding out module 136 may be implemented inall software, all hardware or a combination of software and hardware. Asillustrated, the binding in module 134 receives input messages from theproxy machine 220 in a format associated with the network-basedprocedure call communication of the service 125 and transmits theseinput messages to the network-based procedure call interface 140. Thebinding out module 136 receives messages from the network-basedprocedure call interface 140 and provides these messages to the proxymachine 220. The proxy machine 220 then translates these messages into auniversal format and provides them to the service incompatible client210. In one aspect, the service 125 is a Microsoft service, and themessages passing through the binding in module 134 and the binding outmodule 136 are in RPC.

In one aspect, the proxy machine 220 uses a RPC-filter to setup thegateway interface 150. Once the gateway interface 150 is established orsetup, the gateway interface 150 may also be used to communicate RPCcommands, including OPNUM commands. In one aspect, the network basedprocedure call interface 140 is an RPC server that implements an RPCprotocol to allow commands in messages from the proxy machine 220 to beexecuted on the remote server computing device 160. In one aspect, thecommands in the messages from the proxy machine 220 are bound to anaddress space of the remote server computing device 160. In one aspect,the gateway interface 150 implements Microsoft terminal services gateway(TSGW), Microsoft remote desktop gateway (RDGW) or a similar protocolthat is associated with the operating system of the remote servercomputing device 160. If the gateway interface 150 implements TSGW, thenthe gateway interface 150 may exchange data using OPNUM, a specializedsyntax described in the TSGW specification. The OPNUM operations occurvia a specialized RPC packet filter. RPC messages may be bound to TSGWmessages using data exchanging or another technique of modifying memoryrepresentations of data. The remote server computing device 160 mayinclude a remote desktop protocol (RDP) server as the target host. Theproxy machine 220 may include the functions of the RPC packet filter toallow the client to transmit data to the server. In one aspect, RPCmessages use marshalling for data exchange in conjunction withinterchange data language (IDL). RPC messages may be bound to TSGWmessages using marshalling or another technique of modifying memoryrepresentations of data. The remote server computing device 160 mayinclude a remote desktop protocol (RDP) server as the target host.

The gateway interface 150 may be any gateway interface. For example, thegateway interface 150 may include a common gateway interface, a webserver gateway interface, or a simple common gateway interface. In oneexample, the gateway interface 150 may have two sides, a server side ofthe gateway interface 150 and a client side of the gateway interface150. The server side of the gateway interface 150 may call the clientside of the gateway interface 150, providing environment informationplus a callback function (for the client side of the gateway interface150 to use to convey headers to the server side of the gateway interface150), and receiving content (e.g., from the remote server computingdevice 160) in return.

In one aspect, marshalling may refer to techniques for transforming thememory representation of an object to a data format suitable for storageor transmission. Marshalling may be used when data must be moved betweendifferent parts of a computer program or from one program or machine toanother (e.g., from a client to a server or from a server to a client).In one implementation, marshalling may be similar to serialization andmay be used to communicate to remote objects with an object, in thiscase a serialized object. Marshalling may, in some cases, simplifiescomplex communication, using custom/complex objects to communicateinstead of primitives.

In one aspect, an RPC packet filter may be utilized by to a client(e.g., client 210) not having access to an RPC stack for creating RPCcommands to transmit to the server. A client may provide informationthat the client plans to transmit to the proxy machine 220. The proxymachine 220 may combine the information that the client plans totransmit with a RPC header (e.g., in the session layer, layer 5, of theopen systems interconnection model). As a result, a client that does notmodify, or does not have the ability to modify, the RPC layer in theopen systems interconnection (OSI) model can still be able to populatefields in RPC transmissions via operation of the proxy. Lower layers ofthe OSI model in the RPC packet filter may include pre-filled RPC headerdata. Upper layers of the OSI model in the RPC packet filter may includeclient data. The proxy machine 220 may include the functions of the RPCpacket filter to allow the client to transmit data to the server.

The OSI model may be used to represent data communications andnetworking. In one aspect, the OSI model may include the followinglayers: (1) physical layer, (2) data link layer, (3) network layer, (4)transport layer, (5) session layer, (6) presentation layer, and (7)application layer.

The physical layer (layer 1) may be responsible for the specificationsof the transmission media (e.g., cable, wire, or wireless radio) and theelectrical signal that goes into the transmission media. The physicallayer may include one or more of the following functions: (a) providingthe electrical and physical interface to the network, (b) specifying thetype of medium, or (c) specifying how signals are transmitted within themedium. Example physical layer protocols include IEEE 802.11, IEEE802.15, Bluetooth®, and universal serial bus (USB).

The data link layer (layer 2) may create the entity (the packet orframe) which is put on the transmission media. The data link layer mayinclude one or more of the following functions: (a) controlling accessto the communication channels, (b) controlling the flow of data withinthe communication channels, (c) organizing the data into logical frames,(d) identifying specific computers on the network, or (e) detectingcommunication errors. Example data link layer protocols include FrameRelay and Network Switch.

The network layer (layer 3) may be responsible for getting informationto the correct computer or the correct network. The network layer mayinclude one or more of the following functions: (a) moving informationto the correct address, (b) assembling and disassembling packets orframes, (c) determining addressing and routing, or (d) determining thebest path. Example network layer protocols include Internet Protocol(IP) and AppleTalk.

The transport layer (layer 4) may be responsible for providing extraconnection services including error correction. The transport layer mayinclude one or more of the following functions: (a) controlling dataflow, e.g., slowing down transmissions if buffers are about to overflow,(b) fragmenting and reassembling data, (c) acknowledging successfultransmissions, or (d) correcting faulty transmissions. Example transportlayer protocols include transmission control protocol (TCP) and userdatagram protocol (UDP).

The session layer (layer 5) may be responsible for controlling thesetup, termination, and other mechanisms of a session. In one example, asession may include an agreement to communicate between two entities(e.g., modules or machines). The session layer may include one or moreof the following functions: (a) establishing and maintainingconnections, (b) dealing with name recognition, (c) dealing withsynchronization of data transmissions by placing checkpoints within thedata stream so that, if interrupted, the transmission can take up whereit left off, or (d) handling remote procedure calls, e.g., running aprogram on a remote computer. Example session layer protocols includeRPC, Network Basic Input/Output System (NetBIOS), Session AnnouncementProtocol (SAP), Point-to-Point Tunneling Protocol (PPTP), and SocketSecure (SOCKS).

The presentation layer (layer 6) may be responsible for providing syntaxand grammatical rules for presenting data to the application layer(layer 7), including encoding, decoding, and otherwise converting data.The presentation layer may include one or more of the followingfunctions: (a) maintaining representation of character sets (e.g.,American Standard Code for Information Interchange (ASCII) or Unicode),(b) compressing or decompressing of data, e.g., for a communicationprotocol of the data link layer, (c) encrypting or decrypting of data,e.g., via one or more of the Data Encryption Standard (DES), RSA, orSecure Socket Layer (SSL) protocols, (d) implementing bit ordertranslation, (e) implementing byte order translation, or (f) maintainingfile structure. Example presentation layer protocols include SSL,Transport Layer Security (TLS), External Data Representation (XDR), orMultipurpose Internet Mail Extension (MIME).

The application layer (layer 7) may be responsible for providing networkservices to computing devices (e.g., clients or servers) andapplications running on computing devices. The application layer mayprovide one or more of the following functions: (a) providing aninterface between an operating system of a computing device and anetwork or communication path, or (b) providing network services such asfile transfer, mail services, or terminal emulation. Example applicationlayer protocols include Hypertext Transfer Protocol (HTTP), Secure Shell(SSH), File Transfer Protocol (FTP), Session Initiation Protocol (SIP),Network Time Protocol (NTP), RealTime Transport Protocol (RTP),BitTorrent Protocol, and SPDY.

In one aspect, a tunnel 270 is created between the proxy machine 220 andthe remote desktop computing device 160, facilitating communicationbetween the proxy machine 220 and the remote desktop computing device160. In one aspect the tunnel 270 includes a channel 275 connecting theproxy machine 220 with the remote server computing device 160. In oneaspect, there are multiple remote server computing devices 160 connectedto a single proxy machine 220 using a single tunnel 270 and multiplechannels 275.

In one aspect, the proxy machine 220 of the subject technology mayinclude the following elements:

(1) NLA (network layer authentication) referred to as “credssp”comprising NTLM/Kerberos (NT LAN Manager/Kerberos).

(2) HTTPS/SSL (hypertext transfer protocol secure/secure socketlayer)—encryption based security.

(3) RPC Authentication—provides NLA at RPC layer.

(4) RPC Bridge/Filter—This provides stack elevation and demotion withoutRPC.

(5) Tunnel Creation—by use of the RPC bridge/filter.

(6) Channel Creation—by use of the RPC bridge/filter.

(7) Data Exchange—by use of the RPC bridge/filter.

In one aspect, stack elevation and demotion may refer to using a higherlayer in the OSI model to traffic lower layer data, for example, via atunnel (e.g., tunnel 170 or tunnel 270). Two modules (e.g., the proxymachine 220 and the remote server computing device 160) separated by afirewall (e.g., firewall 120) may use a communication protocol for ahigher layer in the OSI model to traffic data associated with a lowerlayer in the OSI model. For example, Internet Protocol (IP) data(associated with the network layer, layer 3) may be trafficked viaHypertext Transfer Protocol (HTTP, associated with the applicationlayer, layer 7) transmission techniques.

In some implementations of the OSI stack model, lower number layer datamay not be transmitted over higher number layer protocols. Thus, stackelevation and demotion may provide a contradiction to the OSI stack tothe degree that lower layer data may be transmitted via a higher layerprotocol. As a result of stack elevation and demotion, the OSI layerstack may represent a bidirectional stream of data, rather than aunidirectional stream of data, as in some other implementations.

Under typical operation of the OSI model (without stack elevation anddemotion), lower number layers have access to higher number layers, buthigher number layers do not have access to lower number layers. Forexample, under typical operation of the OSI model, the physical layer(layer 1) may know which data link layer (layer 2) protocols areimplemented, but not vice versa. Under stack elevation and demotion, thedata link layer (layer 2) may know which physical layer (layer 1)protocols are being implemented, and vice versa. As a result, the OSIlayer stack may represent a bidirectional stream of data, rather than aunidirectional stream of data (from low number layers to high numberlayers only).

Furthermore, if m and n represent layer numbers of the OSI model stack,where m<n, under stack elevation and demotion, data blocks of layernumber m (lower number layer) may be trafficked via a layer number n(higher number layer) protocol. While in a typical implementation aprotocol at layer n includes protocol IDs for layers n+1 to 7, withstack elevation and demotion from layer m to layer n, where m<n, datafor layer n includes protocol IDs for layers m+1 to 7.

In one aspect of the subject technology, NLA may authenticate the userfor access to the TSGW server; HTTPS/SSL may be used as the fundamentaltransport protocol; and RPC authentication may be a component of the RPCfilter. According to one aspect, it is required to assure that a givenuser has permission to access the RPC layer; RPC Bridge may be themethod of providing a data-path to the logical role/service known asTSGW Tunnel Creation is one example of a command set to TSGW using RPCthat creates an “hole” through a firewall. Channel Creation is oneexample of a command set to TSGW using RPC that creates a logical“channel” to a target host. According to one aspect, it is the channelby which data is transferred to and from a target host within afirewall. Data Exchange may be the process of using a Channel forsending or receiving data through a TSGW server.

In one example, a tunneling protocol (e.g., the tunneling protocol oftunnel 270 and channel 275) may be used when one network protocol (thedelivery protocol) encapsulates a different payload protocol. By usingtunneling, the computer system 200 can, for example, carry a payloadover an incompatible delivery-network, or provide a secure path throughan untrusted network.

In one example, tunneling may contrast with a layered protocol modelsuch as those of OSI or TCP/IP. The delivery protocol may, in somecases, operate at a higher level in the model (e.g., OSI model) thandoes the payload protocol, or at the same level.

Tunneling protocols may use data encryption to transport insecurepayload protocols over a public network (such as the Internet), therebyproviding, e.g., virtual private network (VPN) functionality. Forexample, the IPsec protocol includes an end-to-end transport mode, butcan also operate in a tunneling mode through a trusted security gateway.

In one example, a secure shell (SSH) tunnel may include an encryptedtunnel created through a SSH protocol connection. A SSH tunnel may beused to transfer unencrypted traffic over a network through an encryptedchannel. For example, Microsoft Windows® machines can share files usingthe Server Message Block (SMB) protocol, a non-encrypted protocol. Inone example, in a Microsoft Windows® file-system mounted remotelythrough the Internet, an eavesdropper snooping on the connection couldsee transferred files. However, to mount the Windows® file-systemsecurely, a SSH tunnel that routes all SMB traffic to the remotefileserver through an encrypted channel may be established. Even thoughthe SMB protocol itself may contain no encryption, the encrypted SSHchannel through which the SMB protocol data travels may offer security.

In one example, a SSH tunnel may be set up by configuring a SSH clientto forward a specified local port to a port on the remote machine. Oncethe SSH tunnel has been established, the SSH client can connect to thespecified local port to access the network service. The local port neednot have the same port number as the remote port.

SSH tunnels may provide a means to bypass firewalls that prohibitcertain Internet services—so long as a site allows outgoing connections.For example, an organization may prohibit a user from accessing Internetweb pages (port 80) directly without passing through the organization'sproxy filter (which provides the organization with a means of monitoringand controlling what the user sees through the web). But users may notwish to have their web traffic monitored or blocked by theorganization's proxy filter. If users can connect to an external SSHserver, they can create a SSH tunnel to forward a given port on theirlocal machine to port 80 on a remote web-server.

Some example SSH clients support dynamic port forwarding that allowscreation of a SOCKS 4/5 proxy. As a result, applications can beconfigured to use a local SOCKS proxy server, giving applications moreflexibility than creating a SSH tunnel to a single port. SOCKS can freea client from the limitations of connecting only to a predefined remoteport and server. If an application does not support SOCKS, one can use a“socksifier” to redirect the application to the local SOCKS proxyserver.

In one example, tunneling may be used to “sneak through” a firewall,using a protocol that the firewall would normally block, but is“wrapped” inside a protocol that the firewall does not block, such ashypertext transfer protocol (HTTP). If the firewall policy does notspecifically exclude this kind of “wrapping,” such wrapping can functionto get around the intended firewall policy.

Another example HTTP-based tunneling method uses the HTTP CONNECTapproach. A client may issue the HTTP CONNECT command to a HTTP proxy.The proxy then makes a TCP connection to a particular server port, andrelays data between that server port and the client connection. As HTTPCONNECT may create a security hole, CONNECT-capable HTTP proxies mayrestrict access to the CONNECT approach. The proxy may, in one example,allow access only to a white list of specific authorized servers.

Native Client Tunnel Service

In particular embodiments, a client (e.g., client 210) may be a thinclient primarily including web browsing functionality with CFE trafficgenerated for web-browser. A Native Client tunneling service may beprovided to enable CBE traffic from the client to access hosts throughtunneling (e.g., VPN) mechanisms that may not otherwise be available inbrowser-only environments. The Native Client tunnel service may providea means to create a tunneling protocol (e.g., for VPN) within thebrowser of the client using Native Client conventions found in browsers(e.g., GOOGLE CHROME). The NaCl tunneling service may enable aport-forwarding mechanism that appears to connect to the local host butre-transmits the data via a tunneling protocol to the appropriate targethost (e.g., remote server computing device 160) that may lie behind afirewall (e.g., firewall 120). The NaCl tunneling service may enableend-to-end communication from a browser to a target host within afirewall without requirement of external functions outside of thebrowser.

The NaCl tunneling service may include multiple elements. The NaCltunneling service may include native CHROME library support for anembedded solution. The service may include a dialogue menu for a user tospecify a target host, port-forwarding, tunnel protocol devices, and thetype of tunneling protocol. The service may also include SSH tunnelingsupport through port forwarding or tunneling devices (e.g., to enablelocal connections to connect to a remote host through a remote firewallusing the SSH protocol). The service may also include TSGW tunnelingsupport through port forwarding or tunneling devices (e.g., to enablelocal connections to connect to a remote host through a remote firewallusing the TSGW protocol).

The NaCl tunneling service may perform the following steps. First, anHTML client may load and execute an HTML script with the NaClspecification. The user of the client may specify (e.g., via thedialogue menu) connection information including credentials for thetunnel A port forward or tunnel device may be created. The browser ofthe client may then connect to the target host through the port-forwardor tunnel device. These steps may allow for a local connection thatprovides remote access to a host through a firewall using a tunnelprotocol. This may, for example, provide a full-featured VPNarchitecture from within a browser, enabling access to or from anymachine within a firewall using appropriate tunnel protocols.

Example of Proxy Machine

FIG. 3 illustrates an example of a proxy machine 220 in accordance withone aspect of the disclosure. As shown, the proxy machine 220 includes aprocessor 305, a network interface card 310, and a memory 320 connectedby a bus or interconnect 315.

The processor 305 functions to execute instructions that are provided tothe processor 305 from the memory 320 or other storage locations. Thenetwork interface card 310 allows the proxy machine 220 to connect to anetwork, such as the Internet, a cellular network, a local area network(LAN) or an intranet. The memory 320 stores data and instructions thatcan be implemented or modified by the processor 305.

As shown, the memory 320 includes a setup module 325, a translationmodule 340, client credentials 345, and service credentials 350. Thesetup module 325 is configured to initiate the communication between theservice incompatible client 210, the proxy machine 220, and the service225. As illustrated, the setup module 325 includes a create tunnelmodule 330 and a create channel module 335.

The create tunnel module 330 facilitates the creation of the tunnel 270between the proxy machine 220 and the service 125. The create channelmodule facilitates the creation of channel 275 within the tunnel 270.While only one channel 275 is illustrated, in one aspect, multiplechannels 275 may be created. In one aspect, channel 275 connects theproxy machine 220 with the remote server computing device 160. Theoperation of the setup module is described in more detail in conjunctionwith FIG. 7, below.

The translation module 340 facilitates the translation of messagesbetween the universal format for communication with the serviceincompatible client 210 and the network-based procedure call format ofthe service 125 (e.g. RPC). In one embodiment, the network-basedprocedure call format of the service is RPC and RPC encodings ofcommands and data are stored with the translation module 340 to allowthe translation module to convert messages between the universal formatand RPC.

As shown, the memory 320 further includes client credentials 345 andservice credentials 350. In one aspect, the client credentials 345include a username, a password, a domain name, and a host name. In oneaspect, the service credentials 350 include valid usernames of users ofthe service, valid passwords associated with the usernames, a domainname, and a host name.

The proxy machine 220 and/or a module for setting up the proxy machine220 may reside within (e.g., embedded in) a router for establishing aport forward or a virtual private network (VPN) through a gatewayservice (e.g., a TSGW service or service 125). In some implementations,RPC code may not be available on client devices running non-proprietary(e.g., non-Microsoft) operating systems operating without a proxymachine or running RPC, or such clients may require a large amount ofcode. Hence, in one aspect, having a proxy machine embedded in a routerfor establishing a VPN or port forward through a gateway service iscurrently not possible due to the amount of code required and the factthat the RPC code is not available on non-proprietary (e.g.,non-Microsoft) operating systems. It should be noted that TSGW is oneexample of a gateway protocol. Any other gateway protocol (e.g., aproprietary gateway protocol) may be used in conjunction with thesubject technology. Also, in some aspects, the subject technology mayinclude any modification, augmentation, or omission to the RPC layer.Proxy may be used to translate CFE traffic, CBE traffic, or both CFEtraffic and CBE traffic.

Example of Translation Module

FIG. 4 illustrates an example of a translation module 340 in accordancewith one aspect of the disclosure.

As shown, the translation module 325 includes a request 405 from aclient in a universal format, universal format data 410 to be sent tothe client, encoding 415 for the network-based procedure call interfaceof the service 125, encoding 420 for data to be sent to the service, andencoded data 425 received from the service. In one aspect, thenetwork-based procedure call interface of the service 125 is in RPC.

In one aspect, the translation module 340 receives a request 405 fromthe service incompatible client 210 in a universal format. Thetranslation module 340 uses the encoding 415 for the network-basedprocedure call interface of the service to encode the request 405 in thenetwork-based procedure call communication format, resulting in anencoding 420 for data to be sent to the service 125. In one aspect, thetranslation module 340 then facilitates sending this encoding 420 to theservice 125.

In one aspect, the translation module 340 receives encoded data 425 fromthe service 125. The translation module 340 then decodes the encodeddata 425 from the service 125 based on the encoding 415 for thenetwork-based procedure call interface of the service 125. The result isuniversal format data 410 that can be sent to the service incompatibleclient 210. In one aspect, the translation module 340 then facilitatessending the universal format data 410 to the service incompatible client210.

Example of Computing Device

FIG. 5 is a conceptual block diagram illustrating an example of acomputing device.

A computing device 500 may be, for example, a service compatible client110, a service incompatible client 210, a proxy machine 220, a bindinginterface 130, a RPC server 140, a gateway interface 150 or a remoteserver computing device 160. A computing device may comprise one or morecomputing devices.

A computing device 500 may include a processing system 502. Theprocessing system 502 is capable of communication with a receiver 506and a transmitter 508 through a bus 504 or other structures or devices.It should be understood that communication means other than busses canbe utilized with the disclosed configurations. The processing system 502can generate commands, messages, and/or other types of data to beprovided to the transmitter 509 for communication. In addition,commands, messages, and/or other types of data can be received at thereceiver 506, and processed by the processing system 502.

The processing system 502 may operate in conjunction with ageneral-purpose processor or a specific-purpose processor for executinginstructions and may further include a machine-readable medium 519 forstoring data and/or instructions for software programs. Theinstructions, which may be stored in a machine-readable medium 510and/or 519, are executable by the processing system 502 to control andmanage access to the various networks, as well as provide othercommunication and processing functions. The instructions may alsoinclude instructions executable by the processing system 502 for varioususer interface devices, such as a display 512 and a keypad 514. Theprocessing system 502 may include an input port 522 and an output port524. Each of the input port 522 and the output port 524 may include oneor more ports. The input port 522 and the output port 524 may be thesame port (e.g., a bi-directional port) or may be different ports.

The processing system 502 may be implemented using software, hardware,or a combination of both. By way of example, the processing system 502may be implemented with one or more processors. A processor may be ageneral-purpose microprocessor, a microcontroller, a digital signalprocessor (DSP), an application specific integrated circuit (ASIC), afield programmable gate array (FPGA), a programmable logic device (PLD),a controller, a state machine, gated logic, discrete hardwarecomponents, and/or any other suitable device that can performcalculations or other manipulations of information. Those skilled in theart will recognize how best to implement the described functionality forthe processing system 502.

Software shall be construed broadly to mean instructions, data, or anycombination thereof, whether referred to as software, firmware,middleware, microcode, hardware description language, or otherwise.Instructions may include code (e.g., in source code format, binary codeformat, executable code format, or any other suitable format of code).Instructions may be executable, for example, by a computing device(e.g., a client computing device, an HTTP server, a web server) or by aprocessing system (e.g., an operating system, an HTTP server, or a webserver). Instructions can be, for example, a computer program includingcode.

A machine-readable medium can be one or more machine-readable media. Amachine-readable medium (e.g., 510) may include storage external to anoperating system, such as a random access memory (RAM) 550, a flashmemory 530, a read only memory (ROM) 540, a programmable read-onlymemory (PROM), an erasable PROM (EPROM), registers, a hard disk, aremovable disk, a CD-ROM, a DVD, or any other suitable storage device. Amachine-readable medium 519 may also have a volatile memory and anon-volatile memory. The machine-readable medium 519 may be anon-transitory machine-readable medium. A non-transitorymachine-readable medium may include one or more volatile and/ornon-volatile memories. A machine-readable medium 519 may include storageintegrated into a processing system, such as might be the case with anapplication specific integrated circuit (ASIC). A memory may be amachine-readable medium (e.g., 510 or 519) or a part thereof.

According to one aspect of the disclosure, a machine-readable medium isa computer readable medium encoded or stored with instructions and is acomputing element, which defines structural and functionalinterrelationships between the instructions and the rest of thecomputing device, which permit the instructions' functionality to berealized. In one aspect, a machine-readable medium is a non-transitorymachine-readable medium, a machine-readable storage medium, or anon-transitory machine-readable storage medium. In one aspect, amachine-readable medium is a computer-readable medium, a non-transitorycomputer-readable medium, a computer readable storage medium, or anon-transitory computer-readable storage medium.

An interface 516 may be any type of interface and may reside between anyof the components shown in FIG. 5. An interface 516 may also be, forexample, an interface to the outside world (e.g., an Internet networkinterface). A transceiver block 507 may represent one or moretransceivers, and each transceiver may include a receiver 506 and atransmitter 509. A functionality implemented in a processing system 502may be implemented in a portion of a receiver 506, a portion of atransmitter 509, a portion of a machine-readable medium 510, a portionof a display 512, a portion of a keypad 514, or a portion of aninterface 516, and vice versa. In one aspect, a computing device mayinclude only some or all of the elements shown in FIG. 5. A computingdevice may include other elements not shown in FIG. 5. A computingdevice may include more than one of the same elements.

Example of Network System

FIG. 6 illustrates a simplified diagram of an example of a computernetwork system in accordance with an aspect of the present disclosure.

A computer network system 600 may include one or more client computingdevices 602 (e.g., laptop computers, desktop computers, tablets, PDAs,mobile phones, etc.) in communication with one or more server computingdevices 604 (e.g., a server such as an HTTP server, a web server, anenterprise server, etc.) via a network 606. In one aspect, a servercomputing device 604 is configured to allow remote sessions (e.g.,remote desktop sessions) wherein users can access applications and fileson the server computing device 604 by logging onto the server computingdevice 604 from a client computing device 602. Such a connection may beestablished using any of several well-known techniques such as theremote desktop protocol (RDP) on a Windows-based server or thetechniques disclosed herein for a non-Windows-based server.

In one aspect of the disclosure, a client computing device 602 may be anend-user computing device, such as a laptop or desktop computer. In oneaspect, a server computing device 604 may be a terminal services gateway(TSGW) server or a remote desktop gateway (RDGW) server.

By way of illustration and not limitation, a client computing device 602can represent a computer, a mobile phone, a laptop computer, a tablet, athin computing device, a personal digital assistant (PDA), a portablecomputing device, a virtual machine, or a suitable device with aprocessor. In one example, a client computing device 602 is a smartphone (e.g., iPhone, Android phone, Blackberry, etc.). In certainconfigurations, a client computing device 602 can represent an audioplayer, a game console, a camera, a camcorder, an audio device, a videodevice, a multimedia device, or a device capable of supporting aconnection to a remote computing device. In an advantageous example, aclient computing device 602 is mobile. In another advantageous example,a client computing device 602 is a hand-held device. In another example,a client computing device 602 can be stationary. In one example, aclient computing device 602 may be a device having at least a processorand memory, where the total amount of memory of the client computingdevice 602 is less than the total amount of memory in a server computingdevice 604. In an advantageous example, a client computing device 602does not have a hard disk. In one advantageous aspect, a clientcomputing device 602 has a display smaller than a display supported by aserver computing device 604.

In one aspect, a server computing device 604 may represent a computer, alaptop computer, a computing device, a virtual machine (e.g., VMware®Virtual Machine), a desktop session (e.g., Microsoft Terminal Server), apublished application (e.g., Microsoft Terminal Server) or a suitabledevice with a processor. In one aspect, a server computing device 604can be stationary. In another aspect, a server computing device 604 canbe mobile. In certain configurations, a server computing device 604 maybe any device that can represent a computing device. In one aspect, aserver computing device 604 may include one or more computing devices.

In one example, a first device is remote to a second device when thefirst device is not directly connected to the second device. In oneexample, a first remote device may be connected to a second device overa communication network such as a Local Area Network (LAN), a Wide AreaNetwork (WAN), and/or other network.

When a client computing device 602 and a server computing device 604 areremote with respect to each other, a client computing device 602 mayconnect to a server computing device 604 over a network 606, forexample, via a modem connection, a LAN connection including the Ethernetor a broadband WAN connection including DSL, Cable, T1, T3, FiberOptics, Wi-Fi, or a mobile network connection including GSM, GPRS, 3G,WiMax or other network connection. A network 606 can be a LAN network, aWAN network, a wireless network, the Internet, an intranet or othernetwork. A remote device (e.g., a computing device) on a network may beaddressed by a corresponding network address, such as, but not limitedto, an Internet protocol (IP) address, an Internet name, a WindowsInternet name service (WINS) name, a domain name or other system name.These illustrate some examples as to how one device may be remote toanother device. However, the subject technology is not limited to theseexamples.

Example of Process of Facilitating Creating Tunnel and Channels BetweenProxy Machine and Service

In accordance with one aspect of the disclosure, FIG. 7 illustrates anexample of a process 700 of facilitating creating a tunnel 270 andchannel 275 between the proxy machine 220 and the remote servercomputing device 160.

At process 705, the setup module 325 in the proxy machine 220authenticates the service incompatible client 210. In one aspect, theauthentication is accomplished by verifying the client credentials 345.The client credentials 345 may include a username, a password, a domainname, and a host name. Persons skilled in the art will recognize othertechniques through which the setup module 325 could authenticate theservice incompatible client 210.

At process 710, the setup module 325 in the proxy machine 220authenticates the service. In one aspect, the authentication isaccomplished by verifying the service credentials 350. The servicecredentials 350 may include valid usernames of users of the service,valid passwords associated with the usernames, a domain name, and a hostname.

At process 715, the setup module 325 in the proxy machine 220authenticates the encoding 415 for the network-based procedure callinterface of the service 125 stored in association with the translationmodule 340 in the proxy machine 220. In one aspect, the service 125implements Microsoft Server and the network-based procedure callinterface of the service 125 is Microsoft RPC.

At process 720, the setup module 325 in the proxy machine 220facilitates binding the network-based procedure call data of the service125 to the protocol of the gateway interface 150. In one aspect, theprotocol of the gateway server is TSGW or RDGW.

At process 725, the setup module 325 in the proxy machine 220facilitates verifying the message size of the transmissions to theservice incompatible client 210 and to the service 125. In one aspect,the messages to the service incompatible client are in a universalformat. In one aspect, marshalling is used to ensure that messages fromthe proxy machine 220 to the service 125 are of the appropriate messagesize.

At process 730, the create tunnel module 330 in the setup module 325 inthe proxy machine 220 facilitates creating a tunnel 270 between theproxy machine 220 and the remote server computing device 160. In oneaspect, the tunnel 270 facilitates communication between the proxymachine 220 and the remote server computing device 160 using thenetwork-based procedure call communication of the service 125.

At process 735, the create channel module 335 in the setup module 325 inthe proxy machine 220 facilitates creating channel 275 between the proxymachine 220 and the remote sever computing device 160. In one aspect,there may be multiple remote server computing devices 160 and multiplechannels 275 may be created.

Example of Process of Facilitating Communication Between Client andService

In accordance with one aspect of the disclosure, FIG. 8 illustrates anexample of a process 800 of facilitating communication between theservice incompatible client 210 and the remote server computing device160.

At process 805, the translation module 340 in the proxy machine 220receives data in a universal format from the service incompatible client210. In one aspect, the data is a request 405 from the serviceincompatible client 210. In one aspect the request 405 is received bythe proxy machine 220 over a native IP connection.

At process 810, the translation module 340 in the proxy machine 220encodes the data into an network-based procedure call interfaceassociated with the service 125 based on an encoding 415 for thenetwork-based procedure call interface of the service 125. In oneaspect, the result is an encoding of data 420 to be sent to the service.In one aspect, the format for the network-based procedure callcommunication of the service 125 is RPC an the encoding 415 includes aset of RPC commands and data. In one aspect, the encoding 415 includes aset of commands and data in the network-based procedure call interfaceof the service 125.

At process 815, the translation module 340 in the proxy machine 220facilitates transmitting the data 420 into the network-based procedurecall interface associated with the service 125. In one aspect, the data420 is transmitted to the service 125 over the tunnel 270.

At process 820, the translation module 340 in the proxy machine 220receives a response from the service including response data 425 encodedin the network-based procedure call interface associated with theservice 125. In one aspect, the response data 425 is a response to thedata 420 that was submitted to the service 125. In one aspect, theresponse data 425 is received over the tunnel 270.

At process 825, the translation module 340 in the proxy machine 220decodes the response data into the universal format based on theencoding 415 for the network-based procedure call interface of theservice. In one aspect, the result of the decoding is universal formatdata 410 to be sent to the service incompatible client 210.

At process 830, the translation module 340 in the proxy machine 220facilitates transmitting the universal format data 410 to the serviceincompatible client 210. In one aspect, the universal format data 410 istransmitted to the service incompatible client 210 over a native IPconnection.

In one example, a process of the subject technology is described asfollows:

Setup (Inbound)

(1) NTLM Authentication through HTTP over SSL.

(2) NTLM Authentication through RPC over SSL over HTTP.

(3) HTTP to RPC Bindings.

Setup (Outbound)

(4) NTLM Authentication through HTTP over SSL.

(5) NTLM Authentication through RPC over SSL over HTTP.

(6) HTTP to RPC Bindings.

Setup (RPC)

(7) RPC to TSGW Bindings.

(8) RPC Marshal size (32 bit).

TSGWviaRPC

(9) Operations 1-9 (OPNUM).

One function of the process outlined above may be the setup of aninbound web-service authenticated HTTPS pathway, and the setup ofauthenticated RPC inbound access. The process may also provide the setupof outbound web-service authenticated HTTPS pathway, and the setup ofoutbound authenticated RPC access. The process may also provide thesetup of RPC binding to TSGW role and the setup of RPC marshalling datasize (32 bit fields). The use of OPNUM can provide tunnel creation,tunnel authentication, channel creation, and channel binding. In oneexample, all OPNUM operations occur by use of a specialized RPC packetfilter. These steps may be required to meet Microsoft's requirements foraccessing TSGW role providing this service if the service 125 implementsa Microsoft operating system such as Microsoft Server.

In one aspect, a unique HTTPS connection is created for inbound andoutbound data. These are logically bound (by use of RPC protocol) toestablish a single logical connection to RPC services. These are in turnbound to a TSGW using RPC protocol through a fixed filter mechanism. Theresult is the ability to exchange data using OPNUM. OPNUM arespecialized syntax described by TSGW specification. In one aspect, theserequire RPC pathway from outside firewall to utilize. Once established,a Tunnel and Channel may be created for transporting of data from aclient to a Target server, such as a RDP server. The result achieved maybe a tunnel through the firewall 120 to Microsoft TSGW role with achannel (data path) from/to target host of the remote server computingdevice 160.

In one aspect, commands implementing various RPC commands such as OPNUMare received either by functional passing of parameters (when includedwithin the service compatible client 110 or the proxy machine 220) orthrough protocol syntax. The subject technology can interface thefeatures of the proxy machine 220 through commands by function or byprotocol syntax.

Example of Remote Gateway Client Proxy

FIG. 9 illustrates an example of a client-server computer system 900. Asshown, the computer system 900 includes client computing device 910,firewall 920, binding server 930, remote procedure call (RPC) server940, gateway server 950, and remote server computing device 960.

The client computing device 910 can be a laptop computer, a desktopcomputer, a tablet computer, a personal digital assistant (PDA), amobile phone, or any similar device. The client computing device mayimplement a substantially arbitrary operating system having hypertexttransfer protocol secure (HTTPS) communication capabilities. As shown,the client computing device 910 is connected with the binding server 930over the firewall 920, which may function to create or deny networkconnections based on a set of rules. In one aspect, a unique HTTPSconnection is created for inbound data (directed to the remote servercomputing device 960) and outbound data (directed to the clientcomputing device 910).

As shown, the binding server 930 includes a memory 932. The memory 932includes a binding in module 934 and a binding out module 936. Thebinding in module 934 and binding out module 936 may be implemented inall software, all hardware or a combination of software and hardware. Asillustrated, the binding in module 934 receives input messages from theclient computing device 910 in a format associated with the clientcomputing device, such as HTTPS, and translates the messages into aformat that is associated with the remote server computing device 960,such as a format specific to the operating system of the remote servercomputing device 960. The binding out module 936 receives input messagesfrom the server computing device 960 and translates the messages into aformat that is associated with the client computing device 910.

As illustrated, the output from the binding out module 936 istransmitted to the RPC server 940. The input to the binding in module934 is transmitted from the RPC server 940. In one aspect, the RPCserver 940 implements an RPC protocol to allow commands in messages fromthe client computing device 910 to be executed on the remote servercomputing device 960. In one aspect, the commands in the messages fromthe client computing device 910 are bound to an address space of theremote server computing device 960 using the gateway server 950. In oneaspect, the gateway server 950 implements Microsoft terminal servicesgateway (TSGW), Microsoft remote desktop gateway (RDGW) or a similarprotocol that is associated with the operating system of the servercomputing device 960. If the gateway server 950 implements TSGW, thenthe gateway server 950 may exchange data using OPNUM, a specializedsyntax described in the TSGW specification. In the OPNUM implementation,a tunnel and channel system may be created for transporting data fromthe client computing device 910 to the remote server computing device960. The OPNUM operations occur via a specialized RPC packet filter. RPCmessages may be bound to TSGW messages using marshalling or anothertechnique of modifying memory representations of data. The remote servercomputing device 960 may include a remote desktop protocol (RDP) serveras the target host.

To summarize, according to one aspect, the communication between theclient computing device 910 and the binding server 930 over the firewall920 is implemented in an authenticated HTTPS pathway that is independentof any operating system. The communication between the binding server930 and the RPC server 940 is implemented using the RPC protocol. Thecommunication between the RPC server 940 and the gateway server 950 isin RPC or a remote server computing device 960 operating system specificprotocol, such as OPNUM if the gateway server 950 implements MicrosoftTSGW.

In one aspect, the subject technology may include the followingelements:

(1) NLA (network layer authentication) referred to as “credssp”comprising NTLM/Kerberos (NT LAN Manager/Kerberos).

(2) HTTPS/SSL (hypertext transfer protocol secure/secure socketlayer)—encryption based security.

(3) RPC Authentication—provides NLA at RPC layer.

(4) RPC Bridge/Filter—This provides stack elevation and demotion withoutRPC.

(5) Tunnel Creation—by use of the RPC bridge/filter.

(6) Channel Creation—by use of the RPC bridge/filter.

(7) Data Exchange—by use of the RPC bridge/filter.

In one aspect of the subject technology, NLA may authenticate the userfor access to the TSGW server; HTTPS/SSL may be used as the fundamentaltransport protocol; and RPC authentication may be a component of the RPCfilter. According to one aspect, it is required to assure that a givenuser has permission to access the RPC layer; RPC Bridge may be themethod of providing a data-path to the logical role/service known asTSGW Tunnel Creation is one example of a command set to TSGW using RPCthat creates an “hole” through a firewall. Channel Creation is oneexample of a command set to TSGW using RPC that creates a logical“channel” to a target host. According to one aspect, it is the channelby which data is transferred to and from a target host within afirewall. Data Exchange may be the process of using a Channel forsending or receiving data through a TSGW server.

In accordance with one aspect of the disclosure, FIG. 10 illustrates anexample of a process of translating a message from a format associatedwith a sending computing device to a format understandable to areceiving computing device.

As is understood, the format associated with the sending computingdevice may be HTTPS, and the format understandable to the receivingcomputing device may be RPC. Alternatively, the format associated withthe sending computing device may be RPC, and the format understandableto the receiving computing device may be HTTPS.

At process 1010, the binding out module 936 receives a message from asending computing device, such as remote server computing device 960, ina format that is not understandable to a receiving computing device,such as RPC. At process 1020, the binding out module 936 authenticatesthe message as being from the remote server computing device 960 anddirected to the client computing device 910.

At process 1030, the binding out module 936 translates the message to aformat understandable to the receiving client computing device 910, suchas HTTPS. In one aspect, marshalling is implemented to bring about thetranslation. At process 1040, the binding out module 936 transmits themessage to the receiving client computing device 910.

In an alternative aspect, the binding in module receives a message fromthe sending client computing device 910 in HTTPS or a similar format andtranslated the message to a format understandable to the receivingserver computing device 960, such as RPC, using a procedure similar tothe one described above.

In accordance with one aspect of the disclosure, FIG. 11 illustrates anexample of a process of authenticating a message received in a formatassociated with the client computing device 910 to a format associatedwith the server computing device 960.

At process 1110, the binding in module 934 receives a message in aformat associated with the client computing device 910. In oneimplementation, the message is received from the client computing device910.

At process 1120, the binding in module 934 authenticates the messagefrom the client computing device 910 through a communication protocol,such as HTTPS, over a secure layer, such as secure socket layer (SSL)running in the firewall 920. In one aspect, the authentication may becompleted using NT LAN Manager (NTLM).

At process 1130, the binding in module 934 authenticates the messagethrough a remote procedure call over the secure layer, for example SSL,over the communications protocol, for example HTTPS.

At process 1140, the binding in module 934 binds the communicationprotocol, for example HTTPS, to remote procedure call (RPC) for messagesin the format associated with the client computing device 910.

At process 1150, the binding in module 934 binds the remote procedurecalls to a format associated with the remote server computing device960, for example terminal services gateway (TSGW) if the remote servercomputing device 960 is implementing Microsoft Windows Server.

At process 1160, the binding in module 934 provides the message in aformat associated with the server computing device 960, such as the TSGWformat if the remote server computing device 960 is implementingMicrosoft Windows Server. Marshalling may be used to change the formatof the message.

In accordance with one aspect of the disclosure, FIG. 12 illustrates anexample of a process of authenticating a message received in a formatassociated with the server computing device 960 to a format associatedwith the a client computing device 910.

At process 1210, the binding out module 936 receives a message in aformat associated with the remote server computing device 960. In oneimplementation, the message is received from the remote server computingdevice 960.

At process 1220, the binding out module 936 authenticates the messagefrom the server computing device 960 through a communication protocol,such as HTTPS, over a secure layer, such as SSL.

At process 1230, the binding out module 936 authenticates the messagethrough a remote procedure call over the secure layer, for example SSL,over the communications protocol, for example HTTPS.

At process 1240, the binding out module 936 binds the communicationprotocol, for example HTTPS, to remote procedure call (RPC) for messagesin the format associated with the server computing device 960.

At process 1250, the binding out module 936 binds the remote procedurecall to a format associated with the client computing device 910.

At process 1260, the binding out module 936 provides the message in aformat associated with the client computing device 910, such as theHTTPS format. Marshalling may be used to change the format of themessage.

In one example, a process of the subject technology is described asfollows:

Setup (Inbound)

(1) NTLM Authentication through HTTP over SSL.

(2) NTLM Authentication through RPC over SSL over HTTP.

(3) HTTP to RPC Bindings.

Setup (Outbound)

(4) NTLM Authentication through HTTP over SSL.

(5) NTLM Authentication through RPC over SSL over HTTP.

(6) HTTP to RPC Bindings.

Setup (RPC)

(7) RPC to TSGW Bindings.

(8) RPC Marshal size (32 bit).

TSGWviaRPC

(9) Operations 1-9 (OPNUM).

One function of the process outlined above may be the setup of aninbound web-service authenticated HTTPS pathway, and the setup ofauthenticated RPC inbound access. The process may also provide the setupof outbound web-service authenticated HTTPS pathway, and the setup ofoutbound authenticated RPC access. The process may also provide thesetup of RPC binding to TSGW role and the setup of RPC marshalling datasize (32 bit fields). The use of OPNUM can provide tunnel creation,tunnel authentication, channel creation, and channel binding. In oneexample, all OPNUM operations occur by use of a specialized RPC packetfilter. These steps may be required to meet Microsoft's requirements foraccessing TSGW role providing this service.

In one aspect, a unique HTTPS connection is created for inbound andoutbound data. These are logically bound (by use of RPC protocol) toestablish a single logical connection to RPC services. These are in turnbound to a TSGW using RPC protocol through a fixed filter mechanism. Theresult is the ability to exchange data using OPNUM. OPNUM arespecialized syntax described by TSGW specification. In one aspect, theserequire RPC pathway from outside firewall to utilize. Once established,a Tunnel and Channel may be created for transporting of data from aclient to a Target server, such as a RDP server. The result achieved maybe a tunnel through the firewall 920 to Microsoft TSGW role with achannel (data path) from/to target host of the remote server computingdevice 960.

Example of a Client-Server Communication System

FIG. 13 is a block diagram illustrating an example of a client-servercommunication system 1300.

As shown, the client server communication system 1300 includes a setupengine 1305, a data source 1310, a data sink 1315, a connection engine1320, a remote desktop protocol (RDP) client 1325, an RDP server 1330,and a remote computing device 1335. The setup engine 1305, the datasource 1310, the data sink 1315, the connection engine 1320, the RDPclient 1325, the RDP server 1330, and the remote computing device 1335may be configured to communicate with one another via one or morenetworks, e.g., the Internet, an intranet, a wired or wireless network,etc.

The remote computing device 1335 may be any computing device that may beaccessed remotely via a desktop extension protocol (e.g., RDP, ICA, VNC,or PCoIP protocols), e.g., via a remote desktop session implementingRDP. As shown, the remote computing device 1335 may (but need not)include an operating system 1340. The operating system 1340 may includeone or more applications 1345.1-n that may be executed within theoperating system. The operating system 1340 may include a set ofprograms that manage the hardware resources of the remote computingdevice 1335 and provide common services for the applications 1345.1-n.Please note that applications 1345.1-n may refer to application 1345.1through 1345.n.

The RDP server 1330 may be configured to establish an RDP connection,e.g., a remote desktop session, with the RDP client 1325, to receive RDPinput from the RDP client 1325, to communicate with the remote computingdevice in accordance with the RDP input, and to provide RDP output tothe RDP client 1325. The RDP server 1330 may be implemented in allsoftware, all hardware, or a combination of software and hardware. TheRDP server 1330 may refer to a single physical machine, multiplephysical machines (e.g., a server farm), software residing on a singlephysical machine, or virtualized software residing on a network or “inthe cloud.”

The RDP client 1325 may be configured to establish an RDP connection,e.g., a remote desktop session, with the RDP server 1330, to provide RDPinput to the RDP server 1330, and to receive RDP output from the RDPserver 1330. The RDP client 1325 may be implemented in all software, allhardware, or a combination of software and hardware. The RDP client 1325may refer to a single physical machine, multiple physical machines,software residing on a single physical machine, or virtualized softwareresiding on a network or “in the cloud.”

The data source 1310 may be configured to provide data that may beconverted to CFE input to the RDP client 1325 for communication with theRDP server 1330 through CBE to the remote computing device 1335, e.g.,in a remote desktop session. The data source 1310 may be external to theRDP client 1325. The data source 1310 may be a substantially arbitrarydata source. For example, the data source 1310 may be a computer memory,a physical machine, a virtual machine, a data pipe, etc. The data source1310 may be communicatively coupled with one or more of a keyboard, amouse, a touch screen, a camera, or an audio input unit. The data source1310 may be communicatively coupled with a server in a client-servercommunication protocol identical to or different from RDP. For example,the data source 1310 may be communicatively coupled with a HTTP server.The data source 1310 may be implemented as a physical machine or avirtual machine.

The data sink 1315 may be configured to receive and process dataconverted from CFE output from the RDP client 1325 from communicationwith the RDP server 1330 or the remote computing device 1335, e.g., in aremote desktop session. The data sink 1315 may be external to the RDPclient 1325. The data sink 1315 may be communicatively coupled with oneor more of a video output unit, a display output unit (e.g., an imageoutput unit), or an audio output unit (e.g., any machine with a visualoutput such as a mobile phone or a laptop, or a display, monitor, or aspeaker). The data sink 1315 may be communicatively coupled with aserver in a client-server communication protocol identical to ordifferent from RDP. For example, the data sink 1315 may becommunicatively coupled with a HTTP server and/or a HTTP client. In oneexample, the data sink is communicatively coupled with a HTTP server.The data sink 1315 may also be communicatively coupled with a HTTPclient, e.g., via the HTTP server. The data sink 1315 may be implementedas a physical machine or a virtual machine. The data source 1310 and thedata sink 1315 may reside on the same physical device or on differentphysical devices. The data sink 1315 may be a substantially arbitrarydata sink. The data sink 1315 may be one or more of a video output unit,a display output unit, or an audio output unit. The data sink 1315 maybe a memory unit configured to write received data (e.g., a writeabledisk).

In one aspect, a phrase “defining a data source” may encompass its plainand ordinary meaning, including, but not limited to identifying a datasource (e.g., data source 1310) and/or communicatively coupling the datasource (e.g., via the connection engine 1320) with a protocol (e.g., theRDP protocol implemented with RDP client 1325 and RDP server 1330) fortransmitting data to or communicating with a remote operating system(e.g., operating system 1340 of remote computing device 1335). In oneaspect, communicatively coupling can comprise facilitatingcommunicatively coupling, e.g., providing instructions for communicativecoupling, causing communicatively coupling or enabling communicativelycoupling.

In one aspect, a phrase “defining a data sink” may encompass its plainand ordinary meaning, including, but not limited to identifying the datasink (e.g., data sink 1310) and/or communicatively coupling the datasink (e.g., via the connection engine 1320) with a protocol (e.g., theRDP protocol implemented with RDP client 1325 and RDP server 1330) forreceiving data from or communicating with a remote operating system(e.g., operating system 1340 of remote computing device 1335). In oneaspect, communicatively coupling can comprise facilitatingcommunicatively coupling, e.g., providing instructions for communicativecoupling, causing communicatively coupling or enabling communicativelycoupling.

The connection engine 1320 is configured to connect the data source 1310and the data sink 1315 with the RDP client 1325. The connection engine1320 may be implemented in software. The connection engine 1325 mayreside on a physical machine associated with the RDP client or on adifferent physical machine. The connection engine may reside on the samephysical machine as one or more of the data source 1310 or the data sink1315, or on a physical machine separate and distinct from a physicalmachine where the data source 1310 and/or the data sink 1315 reside.

The connection engine 1320 may be configured to connect to the RDPclient 1325. The connection engine 1320 may also be configured toreceive an indication of the data sink 1315 configured to receive andprocess output data from the RDP communication between the RDP client1325 and the RDP server 1330. The indication of the data sink 1315 mayinclude, for example, a pointer or a link to the data sink 1315 or anetwork address of the data sink 1315. The connection engine 1320 mayalso be configured to receive an indication of the data source 1310 forthe RDP communication between the RDP client 1325 and the RDP server1330. The indication of the data source 1310 may include, for example, apointer or a link to the data source 1310 or a network address of thedata source 1310. The connection engine 1320 may also be configured toreceive an indication of a communication pathway between the data source1310, the data sink 1315, the RDP client 1325, and the RDP server 1330,and to transmit data via the communication pathway. The indication ofthe communication pathway may include, for example, a pointer or a linkto the communication pathway.

The setup engine 1305 may be configured to facilitate creating thecommunication pathway between the data source 1310, the data sink 1315,the RDP client 1325, and the RDP server 1330. In one implementation, thesetup engine may be configured to define an RDP communication protocolbetween the RDP client 1325 and the RDP server 1330. The setup engine1305 may also be configured to define a connection, e.g., of theconnection engine 1320, to the RDP client 1325 and to validate theconnection. The setup engine 1305 may also be configured to define thedata sink 1315 configured to receive and process output data from theRDP communication between the RDP client 1325 and the RDP server 1330.The setup engine 1305 may also be configured to define the data source1310 for the RDP communication between the RDP client 1325 and the RDPserver 1330. The setup engine 1305 may also be configured to create thecommunication pathway between the data source 1310, the data sink 1315,the RDP client 1325, and the RDP server 1330 via the connection engine.The setup engine 1305 may also be configured to facilitate communicationbetween the data source 1310, the data sink 1315, the RDP client 1325,and the RDP server 1330 via the communication pathway. Advantageously,as a result of the client-server communication system 1300 of FIG. 13,data from a substantially arbitrary data source 1310 may be used in RDPcommunications or in remote desktop sessions. Also, data from RDPcommunications or from remote desktop sessions may be output to asubstantially arbitrary data sink 1315. As a result, an operating system1340 of a remote computing device 1335 configured to be accessed via theRDP protocol, may be accessed with arbitrary input data from the datasource 1310 and may provide output data in an arbitrary format to thedata sink 1315. Thus, data received at the RDP client 1325 may be outputto any source, not necessarily a user interface device of a machineimplementing the RDP client 1325. Specifically, in one example, theoperating system 1340 of the remote computing device 1335 may not becompatible with an operating system associated with the data source 1310or the data sink 1315. For example, the operating system 1340 may beassociated with a first operating system family (e.g., MicrosoftWindows®), while the operating system of a machine associated with thedata source 1310 or the data sink may be associated with a secondoperating system family (e.g., Apple Macintosh OS X Snow Leopard®),where the second operating system family is different from and notcompatible with the first operating system family.

In one aspect, a phrase “operating system family,” may encompass itsplain and ordinary meaning including, but not limited to, a group ofoperating systems configured to communicate with one another. A firstexample of an operating system family may include operating systems suchas Microsoft Windows XP®, Microsoft Windows Vista®, Microsoft Windows7®, and Microsoft Windows Server 2008®. A second example of an operatingsystem family, separate and distinct from the first example of anoperating system family, may include operating systems such as AppleMacintosh OS X Snow Leopard® (client), and Apple Macintosh OS X LionServer®.

While the subject technology is described in FIG. 13 in conjunction withan RDP client 1325 and an RDP server 1330 communicating via an RDPconnection, the subject technology may be implemented with anyclient-server communication protocol in place of RDP. In oneimplementation, any screen scraping or remote desktop type protocol mayreplace RDP, as described in FIG. 13. For example, the RDP client 1325and the RDP server 1330, as described in FIG. 13, may be replaced withan independent computing architecture (ICA) client and server, apersonal computer over Internet protocol (PCoIP) client and server, avirtual network computing (VNC) client and server, etc. In one example,the RDP protocol as described in FIG. 13 may be replaced with any otherremote desktop communication protocol. The other remote desktopcommunication protocol mayor may not be developed by MicrosoftCorporation. The other remote desktop communication protocol may be anopen-source protocol or a proprietary protocol. In one example, the RDPprotocol as described in FIG. 13 may be replaced with any applicationlayer protocol in the open systems interconnection (OSI) model forremote desktop communication. In one aspect, a remote desktopcommunication protocol may be a protocol that provides a clientcomputing device (e.g., RDP client 1325) with a graphical interface foraccessing data and/or instructions stored within an operating system(e.g., operating system 1340) of a remote computing device (e.g., remotecomputing device 1335). Using a remote desktop communication protocol, aclient computing device may facilitate providing instructions to aremote computing device to run applications (e.g., one or moreapplications 1345.1-n) within the operating system of the remotecomputing device from the client computing device, as though the userwas operating locally on the remote computing device, rather thanaccessing the remote computing device from the client computing device.In one aspect, a remote desktop communication protocol may be a protocolthat allows control and/or manipulation of a desktop of a remotemachine.

Furthermore, the blocks of FIG. 13 may exist in one or more physicalmachines. For example, a single physical machine may include any, some,or all of the setup engine 1305, the data source 1310, the data sink1315, the connection engine 1320, and the RDP client 1325.Alternatively, each of the setup engine 1305, the data source 1310, thedata sink 1315, the connection engine 1320, and the RDP client 1325 mayreside on separate machines. In one example, the RDP server 1330 and theremote computing device 1335 may reside on the same physical machine. Inanother example, the RDP server 1330 and the remote computing device1335 may reside on separate physical machines.

In FIG. 13, only a single setup engine 1305, only a single data source1310, only a single data sink 1315, only a single connection engine1320, only a single RDP client 1325, only a single RDP server 1330, andonly a single remote computing device 1335 are illustrated. However, thesubject technology may be implemented in conjunction with one or moresetup engines 1305, one or more data source 1310, one or more data sink1315, one or more connection engine 1320, one or more RDP client 1325,one or more RDP server 1330, or one or more remote computing device1335. If there are multiple setup engines 1305, data sources 1310, datasinks 1315, connection engines 1320, RDP clients 1325, RDP servers 1330,or remote computing devices 1335, the multiple setup engines 1305, datasources 1310, data sinks 1315, connection engines 1320, RDP clients1325, RDP servers 1330, or remote computing devices 1335 may reside onthe same physical machine or on separate physical machines.

Example of a Client-Server Communication System Configured to TranscodeData Between Remote Desktop Protocol and Hypertext Transfer Protocol

FIG. 14 is a block diagram illustrating an example of a client-servercommunication system 1400 configured to transcode data between remotedesktop protocol (RDP) and hypertext transfer protocol (HTTP).

As shown, the client-server communication system 1400 includes the datasource 1310, data sink 1315, connection engine 1320, RDP client 1325,RDP server 1330, and remote computing device 1335 of FIG. 13. Inaddition, the client-server communication system 1400 includes a HTTPclient 1405 and a HTTP server 1410. The HTTP server 1410 includes thedata source 1310 and the data sink 1315. HTTP server 1410 may provisionand/or utilize HTTP, HTTPs, WS and/or WSs protocols, or any othersuitable protocols.

The HTTP server 1410 may be connected to the CFE of RDP client 1325 viaa communication pathway through the connection engine 1320. The HTTPserver includes the data source 1310 and the data sink 1315. As aresult, the HTTP server 1410 may provide input data to interact with theremote computing device 1335 and receive output data from interactingwith the remote computing device 1335. The HTTP server 1410 may also beconfigured to communicate with the HTTP client 1405 via an HTTPconnection.

The HTTP client 1405 may be connected to the HTTP server 1410 via anHTTP connection. As a result, the HTTP client may provide data to thedata source 1310 in the HTTP server 1410 and receive data from the datasink 1315 in the HTTP server via a hypertext markup language (HTML)interface.

The connection engine 1320 may be configured to implement the functionsdescribed above. In addition, the connection engine 1320 may beconfigured to transcode data between HTTP and RDP. As a result, the HTTPserver 1410 may be able to receive a representation of the data receivedvia the remote desktop session on the RDP client 1325 and provide datato the RDP client 1325 for communication with the RDP server 1320 or theremote computing device 1335. Thus, the HTTP client 1405 may, via theHTTP server 1410, interface with the RDP client 1325, the RDP server1330, or the remote computing device 1335.

Advantageously, as a result of the client-server communication system1400 of FIG. 14, a HTTP client 1405 running a substantially arbitraryoperating system may interface with an RDP client 1325, an RDP server1330, and a remote computing device 1335 configured to communicate viaRDP. The HTTP client 1405 may run any operating system configured tocommunicate via HTTP and does not need to run an operating systemconfigured to communicate via RDP or run any RDP-specific software.

In one example, the operating system 1340 of the remote computing device1335 may not be compatible with an operating system associated with theHTTP client 1405. For example, the operating system 1340 may beassociated with a first operating system family (e.g., MicrosoftWindows®), while the operating system of the HTTP client 1405 may beassociated with a third operating system family (e.g., Linux Ubuntu®),where the third operating system family is different from and notcompatible with the first operating system family.

While the subject technology is described in FIG. 14 in conjunction withan RDP client 1325 and an RDP server 1330 communicating in a remotedesktop session via an RDP connection, and a HTTP client 1405 and a HTTPserver 1410 communicating via a HTTP connection, the subject technologymay be implemented with any client-server communication protocols inplace of RDP and HTTP. For example, the RDP client 1325 and the RDPserver 1330, as described in FIG. 14, may be replaced with anindependent computing architecture (ICA) client and server, a personalcomputer over Internet protocol (PCoIP) client and server, a virtualnetwork computing (VNC) client and server, etc. In one implementation,the HTTP client 1405 and the HTTP server 1410, as described in FIG. 14,may be replaced with a session initiation protocol (SIP) client andserver. Other protocols may also replace RDP or HTTP as described inFIG. 14. As a result, a client implementing a first client-serverprotocol (e.g., HTTP client 1405) may interface with a serverimplementing a second client-server protocol (e.g., RDP server 1330),where the first client-server protocol may be different from the secondclient-server protocol. In one example, the RDP protocol as described inFIG. 14 may be replaced with any other remote desktop communicationprotocol. The other remote desktop communication protocol mayor may notbe developed by Microsoft Corporation. The other remote desktopcommunication protocol may be an open-source protocol or a proprietaryprotocol. In one example, the RDP protocol as described in FIG. 14 maybe replaced with any application layer protocol in the open systemsinterconnection (OSI) model for remote desktop communication.

Example of an Operation of a Setup Engine

FIG. 15A is a flow chart illustrating an example of a process 1500A thatmay be executed by a setup engine (e.g., setup engine 1305).

The process 1500A begins at operation 1510A, where the setup engine maydefine a first client-server communication protocol (e.g., RDP) betweena client side (e.g., RDP client 1325) and a server side (e.g., RDPserver 1330). The first client-server communication protocol may becommunicatively coupled to an operating system (e.g., operating system1340 of remote computing device 1335) running at least one application(e.g., applications 1345.1-n). The first client-server communicationprotocol may be an example of an application layer protocol in the opensystems interconnection (OSI) model. Defining the communication protocolmay include: selecting the specific application layer protocol for use(e.g., RDP).

In operation 1520A, the setup engine may define a connection for theclient side (e.g., RDP client 1325), including the connection for theserver side (e.g., RDP server 1330) and the connection between theclient side and the server side in the first client-server communicationprotocol. The client side may be connected to a connection engine (e.g.,connection engine 1320). For example, a connection may includecommunicatively coupling a connection engine to a client side. Theconnection engine may be configured to communicate with the client sidevia a network or via a direct wired or wireless connection. Defining aconnection may, for example, include providing instructions as to how toprovide or set up a communicative coupling, creating or facilitatingcreating the communicative coupling, or initiating the communicativecoupling. Defining a connection may also include selecting specificprotocols to use for the layers of the OSI model other than theapplication layer (i.e., protocols for one or more of the physicallayer, data link layer, network layer, transport layer, session layer,and presentation layer). Defining a connection for a client side (e.g.,including connection for a server side and connection between a clientand a server) may include selecting or facilitating selecting aspecified client (e.g., RDP client 1325) and/or a specified server(e.g., RDP server 1330) for the connection or communicative coupling.Two or more devices or modules may be communicatively coupled to oneanother if they are configured to communicate with one another. Forexample, two devices communicating with one another, e.g., via a networkconnection or via a direct wired or wireless connection, may becommunicatively coupled to one another. Within a single computer system,a first module that receives input from a second module and/or providesoutput to the second module may be communicatively coupled with thesecond module.

In operation 1530A, the setup engine may define validation informationthat can be used to validate the connection to the client side and/or tothe service side in the first client-server communication protocol. Forexample, the setup engine may define authentication information to beused to authenticate the connection to the client side. In one aspect, aphrase “validate a connection” may encompass its plain and ordinarymeaning, including, but not limited to verifying that a connection isauthorized and that data may be transmitted via the connection. In someimplementations, validating a connection may include one or more of:verifying login information associated with the connection, verifying anetwork address of at least one terminal (e.g., the client side or theserver side) of the connection, verifying that a virus scan was executedon at least one terminal of the connection, verifying that at least oneterminal of the connection is turned on and connected to the network,etc.

In operation 1540A, the setup engine defines a data sink (e.g., datasink 1315) for the first client server communication protocol. The datasink may be external to the client side and external to the server side(e.g., the data sink may reside on a HTTP server 1410). The data sinkmay be configured to receive and process an output that is received atthe data sink from the client side or the server side within the firstclient-server communication protocol. The output from the firstclient-server communication protocol may include output based on atleast one application (e.g., applications 1345.1-n) running within anoperating system (e.g., operating system 1340) of a remote computingdevice (e.g., remote computing device 1335) coupled to the server side.The data sink may be implemented as a virtual machine or a physicalmachine.

In operation 1550A, the setup engine defines a data source (e.g., datasource 1310) for the first client-server communication protocol. Thedata source may be external to the client side and external to theserver side (e.g., the data source may reside on a HTTP server 1410).The data source may be configured to provide an input to the client sideor the server side in the first client-server communication protocol.The input to the client side or the server side in the firstclient-server communication protocol may include input to a remotecomputing device (e.g., at least one application running within theoperating system of the remote computing device coupled to the serverside). The data source may be implemented as a virtual machine or aphysical machine. The data source may reside on the same physical deviceas the data sink or on a different physical device than the data sink.

In one example, both the data source and the data sink may reside withina HTTP server (e.g., HTTP server 1410). The HTTP server may becommunicatively coupled to a HTTP client (e.g., HTTP client 1405). Thedata source may be configured to receive input from the HTTP client andthe data sink may be configured to provide output to the HTTP client. Inanother example, the HTTP server and the HTTP client may be replacedwith a server and client in any client-server communication protocol,for example, a session initiation protocol (SIP) server and client.

In operation 1560A, the setup engine facilitates creating acommunication pathway between the data source, the data sink, the clientside, and/or the server side. The communication pathway may include aconnection engine (e.g., connection engine 1320).

In operation 1570A, the setup engine facilitates communication betweenthe data source, the data sink, the client side, and/or the server sidevia the communication pathway. If the data source and the data sink areassociated with a second client-server communication protocol (e.g.,HTTP or SIP), facilitating communication via the communication pathwaymay involve transcoding between a format associated with the firstclient-server communication protocol (e.g., RDP) and a format associatedwith the second client-server communication protocol (e.g., HTTP orSIP). For example, the connection engine may transcode between theformat associated with the first client-server communication protocoland the format associated with the second client-server communicationprotocol. After operation 1570A, the process 1500A may be closed.

In some aspects, the setup engine may close the communication pathwayand free (e.g., make available for other purposes) resources consumed bythe communication pathway upon termination of a communication sessionbetween the data source, the data sink, the client side, and/or theserver side.

Example of Instructions for Operation of a Setup Engine

FIG. 15B is an example of a device 1500B including instructions for anoperation of a setup engine (e.g., setup engine 1305). A device 1500Bmay include one or more devices. A device 1500B can be, for example, oneor more machine-readable medium such as one or more memories. A device1500B can be, for example, one or more circuits and/or one or moreprocessors.

The device 1500B may include instructions 1510B to define a firstclient-server communication protocol between a client-side and a serverside.

The device 1500B may also include instructions 1520B to define aconnection between the client side and the server side in the firstclient-server communication protocol.

The device 1500B may also include instructions 1530B to definevalidation information to be used to validate a connection in the firstclient-server communication protocol.

The device 1500B may also include instructions 1540B to define a datasink for the first client-server communication protocol. The data sinkmay be external to the client side and external to the server side. Thedata sink may be configured to receive and process an output from thefirst client-server communication protocol.

The device 1500B may also include instructions 1550B to define a datasource for the first client-server communication protocol. The datasource may be external to the client side and external to the serverside. The data source may be configured to provide an input to the firstclient-server communication protocol.

The device 1500B may also include instructions 1560B to create acommunication pathway between the data source, the data sink, the clientside, and the server side.

The device 1500B may also include instructions 1570B to facilitatecommunication between the data source, the data sink, the client side,or the server side via the communication pathway.

Example of Modules for Operation of a Setup Engine

FIG. 15C is an example of a device 1500C including one or more modulesfor an operation of a setup engine (e.g., setup engine 1305). A device1500C may include one or more devices. A device 1500C can be, forexample, one or more machine-readable medium such as one or morememories. A device 1500C can be, for example, one or more circuitsand/or one or more processors. A module may be a component of a device.

The device 1500C may include a module 1510C for defining a firstclient-server communication protocol between a client-side and a serverside.

The device 1500C may also include a module 1520C for defining aconnection between the client side and the server side in the firstclient-server communication protocol.

The device 1500C may also include a module 1530C for defining validationinformation to be used to validate a connection in the firstclient-server communication protocol.

The device 1500C may also include a module 1540C for defining a datasink for the first client-server communication protocol. The data sinkmay be external to the client side and external to the server side. Thedata sink may be configured to receive and process an output from thefirst client-server communication protocol.

The device 1500C may also include a module 1550C for defining a datasource for the first client-server communication protocol. The datasource may be external to the client side and external to the serverside. The data source may be configured to provide an input to the firstclient-server communication protocol.

The device 1500C may also include a module 1560C for creating acommunication pathway between the data source, the data sink, the clientside, and the server side.

The device 1500C may also include a module 1570C for facilitatingcommunication between the data source, the data sink, the client side,or the server side via the communication pathway.

Example of an Operation of a Connection Engine

FIG. 16A is a flow chart illustrating an example of a process 1600A thatmay be executed by a connection engine (e.g., connection engine 1320).

The process 1600A begins at operation 1610A, where the connection engineconnects to a client side (e.g., RDP client 1325) in a firstclient-server communication protocol (e.g., RDP) between the client sideand a server side (e.g., RDP server 1330).

In operation 1620A, the connection engine receives an indication of adata sink (e.g., data sink 1315) for the first client-servercommunication protocol. The indication of the data sink may include, forexample, a network address of the data sink or a link or a pointer tothe data sink. The data sink may be external to the client side andexternal to the server side. The data sink may be configured to receiveand process output data for the first client-server communicationprotocol. The data sink may be communicatively coupled with a videooutput unit or an audio output unit. The data sink may be implemented asa virtual machine or a physical machine.

In operation 1630A, the connection engine receives an indication of adata source (e.g., data source 1310) for the first client-servercommunication protocol. The indication of the data source may include,for example, a network address of the data source or a link or a pointerto the data source. The data source may be external to the client sideand external to the server side. The data source may be configured toprovide input data to the first client-server communication protocol.The data source may be communicatively coupled with one or more of akeyboard, a mouse, a touch screen, a camera, or an audio input unit. Thedata source may be implemented as a virtual machine or a physicalmachine. The data source may reside on the same physical device as thedata sink or on a different physical device than the data sink.

In one example, the first client-server communication protocol may becommunicatively coupled to an operating system (e.g., operating system1340 of remote computing device 1335) running at least one application(e.g., applications 1345.1-n). The input may include an input to theapplication(s) and the output may include an output from theapplication(s).

In one example, the data source and the data sink may reside within ahypertext transfer protocol (HTTP) server (e.g., HTTP server 1410). TheHTTP server may be communicatively coupled with a HTTP client. The datasource may be configured to receive the input from the HTTP client andthe data sink may be configured to provide the output to the HTTPclient.

In operation 1640A, the connection engine receives an indication of acommunication pathway between the data source, the data sink, the clientside, and/or the server side.

In one implementation, the data source and the data sink may beassociated with any second client-server communication protocol (e.g.,HTTP or session initiation protocol [SIP]), and the communicationpathway may include a transcoder between a format associated with thefirst client-server communication protocol (e.g., RDP), and a formatassociated with the second client-server communication protocol (e.g.,HTTP or SIP).

In operation 1650A, the connection engine facilitates transmitting theinput data from the data source to the client side via the communicationpathway. For example, the connection engine may include code fortransmitting the input data from the data source to the client side. Theconnection engine may cause the data source and/or the client side toexecute the code for transmitting the input data from the data source tothe client side. The input data may be transmitted from the data sourceto the client side via the connection engine.

In operation 1660A, the connection engine facilitates transmission ofthe input data from the client side to the server side. For example, theconnection engine may include code for transmitting the input data fromthe client side to the server side, so that the input data could beprovided to the remote computing device. The connection engine may causethe server side and/or the client side to execute the code fortransmitting the input data from the client side to the server side.

In operation 1670A, the connection engine facilitates transmission ofthe output data from the server side to the client side. For example,the connection engine may include code for transmitting the output datafrom the server side to the client side, so that the output data couldbe provided to the connection engine. The connection engine may causethe server side and/or the client side to execute the code fortransmitting the output data from the server side to the client side.

In operation 1680A, the connection engine facilitates transmitting theoutput data from the client side to the data sink via the communicationpathway. For example, the connection engine may include code fortransmitting the output data from the client side to the data sink. Theconnection engine may cause the client side and/or the data sink toexecute the code for transmitting the output data from the client sideto the data sink. The output data may be transmitted from the clientside to the data sink via the connection engine. After operation 480A,the process 400A may be closed.

Example of Instructions for Operation of a Connection Engine

FIG. 16B is an example of a device 1600B including instructions for anoperation of a connection engine. A device 1600B may include one or moredevices. A device 1600B can be, for example, one or moremachine-readable medium such as one or more memories. A device 1600B canbe, for example, one or more circuits and/or one or more processors.

The device 1600B may include instructions 1610B to connect to a clientside in a first client-server communication protocol between the clientside and a server side.

The device 1600B may also include instructions 1620B to receive anindication of a data sink for the first client server communicationprotocol. The data sink may be external to the client side and externalto the server side. The data sink may be configured to receive andprocess output data for the first client-server communication protocol.

The device 1600B may also include instructions 1630B to receive anindication of a data source for the first client-server communicationprotocol. The data source may be external to the client side andexternal to the server side. The data source may be configured toprovide input data to the first client-server communication protocol.

The device 1600B may also include instructions 1640B to receive anindication of a communication pathway between the data source, the datasink, the client side, or the server side.

The device 1600B may also include instructions 1650B to transmit theinput data from the data source to the client side via the communicationpathway.

The device 1600B may also include instructions 1660B to facilitatetransmission of the input data from the client side to the server side.

The device 1600B may also include instructions 1670B to facilitatetransmission of the output data from the server side to the client side.

The device 1600B may also include instructions 1680B to transmit theoutput data from the client side to the data sink via the communicationpathway.

Example of Modules for Operation of a Connection Engine

FIG. 16C is an example of a device 1600C including one or more modulesfor an operation of a connection engine. A device 1600C may include oneor more devices. A device 1600C can be, for example, one or moremachine-readable medium such as one or more memories. A device 1600C canbe, for example, one or more circuits and/or one or more processors. Amodule may be a component of a device.

The device 1600C may include a module 1610C for connecting to a clientside in a first client-server communication protocol between the clientside and a server side.

The device 1600C may also include a module 1620C for receiving anindication of a data sink for the first client server communicationprotocol. The data sink may be external to the client side and externalto the server side. The data sink may be configured to receive andprocess output data for the first client-server communication protocol.

The device 1600C may also include a module 1630C for receiving anindication of a data source for the first client-server communicationprotocol. The data source may be external to the client side andexternal to the server side. The data source may be configured toprovide input data to the first client-server communication protocol.

The device 1600C may also include a module 1640C for receiving anindication of a communication pathway between the data source, the datasink, the client side, or the server side.

The device 1600C may also include a module 1650C for transmitting theinput data from the data source to the client side via the communicationpathway.

The device 1600C may also include a module 1660C for facilitatingtransmission of the input data from the client side to the server side.

The device 1600C may also include a module 1670C for facilitatingtransmission of the output data from the server side to the client side.

The device 1600C may also include a module 1680C for transmitting theoutput data from the client side to the data sink via the communicationpathway.

Example Virtualization System

FIG. 17 is a block diagram of a local device virtualization system 1700,according to an aspect of the disclosure. The system 1700 may includethe client 1702 in communication with the server 1704, for example, overa network (as illustrated in detail in FIG. 6). The client 1702 mayinclude a proxy 1710, a stub driver 1720, and a bus driver 1730. Theclient 1702 can be connected to a device 1740, as shown in FIG. 17. Theserver 1704 may include an agent 1750, and a virtual bus driver 1760.

According to the illustrated configuration, while the device 1740 is notlocally or physically connected to the server 1704 and is remote to theserver 1704, the device 1740 appears to the server 1704 as if it islocally connected to the server 1704, as discussed further below. Thus,the device 1740 appears to the server 1704 as a virtual device 1790. Inone implementation, one or more of the data source 1310 or the data sink1315 of FIG. 13 may be implemented as a virtual device (e.g., virtualdevice 1790).

By way of illustration and not limitation, the device 1740 may be amachine-readable storage medium (e.g., flash storage device), a printer,a scanner, a camera, a facsimile machine, a phone, an audio device, avideo device, a peripheral device, or other suitable device that can beconnected to the client 1702. The device 1740 may be an external device(i.e., external to the client 1702) or an internal device (i.e.,internal to the client 1702).

In one aspect of the disclosure, the device 1740 is a Universal SerialBus (USB) device that can be locally connected to the client 1702 usinga wired USB or wireless USB connection and communicates with the client1702 according to a USB communications protocol. In another aspect, thedevice 1740 may be a device other than a USB device.

As used herein, a “local” device of a system, or a device “locally”connected to a system, may be a device directly connected to the systemusing one or more wires or connectors (e.g., physically connected to thesystem), or a device directly connected to the system using a wirelesslink (e.g., Bluetooth). For example, device 1740 is a local device ofclient 1702. Furthermore, in one aspect of the disclosure, a localdevice of a system or a device locally connected to a system may includea device within the system (e.g., an internal device of client 1702).

A “remote” device, or a device “remote” to a system, may be a devicethat is not directly connected to the system. For example, the server1704 is remote to both client 1702 and device 1740 because server 1704is not directly connected to client 1702 or device 1740 but connectedindirectly through network 606 (illustrated in FIG. 6), which caninclude, for example, another server, or the Internet.

The bus driver 1730 can be configured to allow the operating system andprograms of the client 1702 to interact with the device 1740. In oneaspect, when the device 1740 is connected to the client 1702 (e.g.,plugged into a port of the client 1702), the bus driver 1730 may detectthe presence of the device 1740 and read information regarding thedevice 1740 (“device information”) from the device 1740. The deviceinformation may include features, characteristics and other informationspecific to the device. For an example of a USB device, the deviceinformation may comprise a device descriptor (e.g., product ID, venderID and/or other information), a configuration descriptor, an interfacedescriptor, an endpoint descriptor and/or a string descriptor. The busdriver 1730 may communicate with the device 1740 through a computer busor other wired or wireless communications interface.

In one aspect, a program (e.g., application) running locally on theclient 1702 may access the device 1740. For example, the device 1740 maybe accessed locally when the client 1702 is not connected to the server1704. In this aspect, the operating system (e.g., Microsoft Windows®) ofthe client 1702 may use the device information to find and load anappropriate device driver (not shown) for the device 1740. The devicedriver may provide the program with a high-level interface to the device1740.

In one aspect, the device 1740 may be accessed from the server 1704 asif the device were connected locally to the server 1740. For example,the device 1740 may be accessible from the desktop running on the server1704 (i.e., virtual desktop environment). In this aspect, the bus driver1730 may be configured to load the stub driver 1720 as the defaultdriver for the device 1740. The stub driver 1720 may be configured toreport the presence of the device 1740 to the proxy 1710 and to providethe device information (e.g., device descriptor) to the proxy 1710.

The proxy 1710 may be configured to report the presence of the device1740, along with the device information, to the agent 1750 of the server1704 over the network 606 (illustrated in FIG. 6). Thus, the stub driver1720 redirects the device 1740 to the server 1704 via the proxy 1710.

The agent 1750 may be configured to receive the report from the proxy1710 that the device 1740 is connected to the client 1702 and the deviceinformation. The agent 1750 can provide notification of the device 1740,along with the device information, to the virtual bus driver 1760. Thevirtual bus driver 1760 may be configured to report to the operatingsystem of the server 1704 that the device 1740 is connected and toprovide the device information to the operating system. This allows theoperating system of the server 1704 to recognize the presence of thedevice 1740 even though the device 1740 is connected to the client 1702.The operating system of the server 1704 may use the device informationto find and load an appropriate device driver 1780 for the device 1740at the server 1704, an example of which is illustrated in FIG. 17. As aresult, the device 1740 is enumerated on the server 1704. Once thepresence of the device 1740 is reported to the operating system of theserver 1704, the device 1740 may be accessible from the desktop runningon the server 1704 (i.e., virtual desktop environment). For example, thedevice 1740 may appear as an icon on the virtual desktop environmentand/or may be accessed by applications running on the server 1704.

In one aspect, an application 1770 running on the server 1704 may accessthe device 1740 by sending a transaction request for the device 1740 tothe virtual bus driver 1760 either directly or through the device driver1780. The virtual bus driver 1760 may direct the transaction request tothe agent 1750, which sends the transaction request to the proxy 1710over the network 106. The proxy 1710 receives the transaction requestfrom the agent 1750, and directs the received transaction request to thestub driver 1720. The stub driver 1720 then directs the transactionrequest to the device 1740 through the bus driver 1730.

The bus driver 1730 receives the result of the transaction request fromthe device 1740 and sends the result of the transaction request to thestub driver 1720. The stub driver 1720 directs the result of thetransaction request to the proxy 1710, which sends the result of thetransaction request to the agent 1750 over the network 106. The agent1750 directs the result of the transaction request to the virtual busdriver 1760. The virtual bus driver 1760 then directs the result of thetransaction request to the application 1770 either directly or throughthe device driver 1780.

Thus, the virtual bus driver 1760 may receive transaction requests forthe device 1740 from the application 1770 and send results of thetransaction requests back to the application 1770 (either directly orthrough the device driver 1780). As such, the application 1770 mayinteract with the virtual bus driver 1760 in the same way as a busdriver for a device that is connected locally to the server 1704. Thevirtual bus driver 1760 may hide the fact that it sends transactionrequests to the agent 1750 and receives the results of the transactionrequests from the agent 1750 instead of a device that is connectedlocally to the server 1704. As a result, the device 1740 connected tothe client 1702 may appear to the application 1770 as if the physicaldevice 1740 is connected locally to the server 1704.

In one implementation, one or more of the data source 1310 or the datasink 1315 of FIG. 13 may be implemented as a virtual device (e.g.,virtual device 1790). The remote computing device 1335 may also beimplemented as a virtual device. Alternatively, one or more of the datasource 1310, the data sink 1315, or the remote computing device 1335 maybe implemented as a physical device (e.g., device 1740). In oneimplementation, the operating system 1340 may be implemented as avirtual operating system running within a virtual device (e.g., virtualdevice 1790) and not tied to any physical device. The server 1704 mayinclude the RDP server 1330 or the HTTP server 1410. The client 1702 mayinclude the RDP client 1325 or the HTTP client 1405. In one example, theclient 1702 may include one or more of items 1305, 1310, 1315, 1320, or1325.

Generic Client Engine

RDP, ICA, VNC, PCoIP are examples of protocols that provide similarfeatures. However, the subject technology may include the ability toload any component of any of these protocols on demand (e.g., using asetup engine 1305 and a connection engine 1320 that are components of ageneral client engine), not to define a client using anyone orcombination of these. A client computing device that does not implementRDP client code may be configured to communicate with an RDP serverusing the techniques described herein.

In some implementations, RDP, ICA, PCoIP and similar protocols, may beimplemented with a client including, for example, a screen, a keyboard,a mouse and a network interface. The client may be connected to a serverproviding operating system (OS) extension to the client by use of aprotocol trafficking same. Some aspects of the subject technology mayinclude a generic component engine that allows for each layer ofoperation to be loaded at the appropriate time, for the appropriateprotocol. Some implementations of the subject technology may include thefollowing.

(1) A protocol is utilized between a client and a server; for thiscomponent, a protocol layer is loaded. (E.g., the communication protocoldescribed in operation 1610A of FIG. 16A and the correspondingdisclosure.)

(2) A parser removes from the protocol screen components and sends themto a rendering routine. (The rendering routine may be within the datasink 1315, the communication with the data sink is described in steps1620A, 1640A, and 1650A-1680A of FIG. 16A and the correspondingdisclosure.)

(3) Keyboard, mouse and other user input are transmitted back to serviceover protocol to server. (The keyboard, mouse, and other user inputs maybe within the data source 1310, the communication with the data sourceis described in steps 1630A, 1640A, and 1650A-1680A of FIG. 16A and thecorresponding disclosure.)

(4) Entire client-side protocol is attached to devices and functions(such as screen, audio in, audio out, smart-card reader, etc) through acommunication queue. (A communication queue may be a portion of thecommunication pathway of steps 440A-480A of FIG. 16A and thecorresponding disclosure.)

(5) Communication queue enables cascade of protocols where eachsuccessive stage uses mating to client-side-queue (CSQ). (Acommunication queue may be a portion of the communication pathway ofsteps 440A-480A of FIG. 16A and the corresponding disclosure. Protocolsmay include protocols within the application layer of the open systeminterconnection model, for example, RDP.)

(6) Client-Complementary queue (CCQ) is attached to CSQ to terminatei/o; CSQ connects device(s) streams to client-side-codec/protocol. Thiscan be written as follows: [N]DevicesCCQ::CSQclient-side-codec::server.(Devices may include input devices within the data source 1310 or outputdevices within the data sink 1315. Connecting devices is described insteps 1620A, 1630A, and 1650A-1680A of FIG. 16A and the correspondingdisclosure.)

An N devices sourcing/sinking data through RDP client (RDPc) through RDPserver (RDPs) RDP client can be written as follows:[N]DevicesCCQ::CSQRDPc::RDPs

Use of CCQ/CSQ is implicit. Therefore, the same representation may bere-written as follows: [N]Devices::RDPc::RDPs

That is, “N” devices are connected via CCQ/CSQ to RDPc and connected toremote RDPs.

(7) Creation of RDPc (e.g., RDP client 1325) and operating connection toRDPs (e.g., RDP server 1330) is communicated through RDPc creationservice (RDPcS). The role of RDPcS is to create RDPc and attach to aspecified RDPs using appropriate protocols OP, tunnel, credentials . . .) as needed, and attach Devices as data source/sink (e.g., data source1310 and data sink 1315) through RDPc instance. The ability to specifyDevices and RDPs for each RDPc provides a dynamic means of creatingclients and routing data traffic in its various forms to the propertarget/source. (See, e.g., steps 420A, 430A, and 450A-480A of FIG. 16Aand the corresponding disclosure.)

Other functions, redirections and features are managed in a similarfashion. Each may be logically for the same function or purpose. In oneaspect, implementation, however, is protocol specific and the entirechain of responsibilities is customized for a given protocol.

In some implementations, a unique personality may be loaded for eachprotocol supported. No generic stack layer loads components andinterprets operation differently as dictated or featured by a protocol.Moreover, the ability to utilize a remote desktop protocol as a genericcodec (or compound codec) is not current art.

Cost may be higher if more than one personality (protocol) is supported.Code foot print may be larger if multiple client applications need toreside locally. Having the ability to dynamically load components on an“as needed/required” basis simplifies overall design. A client may haveonly screen, keyboard and USB port (or less). Code may be downloaded toprovide the CODEC needed to translate data to screen. Moreover, theability add device support continues through USB peripheral expansionand code exchange specifying actual Devices being used by the softclient (RDPc). Once the device i/o are attached, data communication toan RDPs may take place. However, one advantage of some implementationsis the ability to create multiple clients on a remote host. The Devicepaths may be abstracted and data exchange for these trafficked throughentirely different pathways than conventional pathways. This is a truly“virtual client” architecture.

Some aspects may provide simplified method to receive, interpret andexchange data in a universal protocol exchange environment. Once inplace, the operation may utilize other specifications or determineoperation features at run-time. This may be specified explicitly by thecreation of the RDPc, connection to RDPs, and termination/sourcing ofdata through Device(s).

The subject technology may include a specification defining a baseprotocol, or a runtime “sniffer” that pre-reads opening packets todetermine target client features. Once determined, a specification isloaded that provides network compatibility with the serving protocol.Another specification defines means to unpack data stream and divide torespective targets (screen, speakers, etc). Specifications define allinput and output data handlers. Moreover, specifications dictate originof specification information (method and location). Another embodimentwould have a generic “protocol service” that enables an application tospecify the desired protocol, connection, credentials, devicetermination/sourcing and the like. Moreover, a means to perform anoption exchange for the validation or selection of ideal deviceinterfaces and encoding mechanisms, as well as providing for time-stampinformation for aging data.

Some implementations of the subject technology may include thefollowing.

(1) Define protocol (or codec such as RDPc). (See, e.g., operation 310Aof FIG. 15A and its corresponding disclosure.)

(2) Define connection (RDPs) to connect. (E.g., Define an RDP connectionbetween RDP client 1325 and RDP server 1330; see, e.g., operation 320Aof FIG. 15A and its corresponding disclosure.)

(3) Define credentials and other connection/user/machine validationinformation. (See, e.g., operation 330A of FIG. 15A and itscorresponding disclosure.)

(4) Define Devices, their respective encodings and target paths. (a)Define receiver extraction (screen, sound-out, etc). (b) Define senderextraction (keyboard, mouse, touch, camera, sound-in, etc.). (E.g.,define a data source 1310 and a data sink 1315, e.g., as in steps340A-350A of FIG. 15A and its corresponding disclosure; E.g., define orfacilitate defining encodings and/or target paths for various devices inthe data source 1310 and the data sink 1315 for communication via thecommunication pathway as shown in steps 360A and 370A of FIG. 15A andits corresponding disclosure.)

(5) Create instance (RDPc) with Device stream data attachments. (E.g.,create an instance of a communication session via the communicationpathway, where the devices in the data source 1310 and the data sink1315 are communicatively coupled with the RDP client 1325. In oneimplementation, the RDP client 1325 may behave as though the data source1310 and the data sink 1315 are its own input/output devices. E.g., theRDP client 1325 may behave as though the data source 1310 includes itskeyboard and mouse and the data sink 1315 includes its screen and audiospeakers.)

(6) Use communication pathway. (See, e.g., Operation 370A of FIG. 15Aand the corresponding disclosure.)

(7) Close the operation. (E.g., After operation 370A, the operation 300Amay be closed.)

Defining the communication protocol may be useful to establish means ofextraction and insertion of data streams. Defining receiver/senderextractions may be useful to enable data sourcing and sinking. Closingthe operation may be useful to shutdown the specification createdclient.

A specification defined client, or generic client engine (GCE) may beprovided.

Example of Connection Engine

FIG. 18 illustrates an example connection engine 1800. The connectionengine 1800 may correspond to the connection engine 1320 of FIGS. 1 and2, in some examples.

As shown, the connection engine 1800 may include one or more inputdevices 1805.1 through 1805.4, an input translation block 1810, anoutput translation block 1820, and one or more output devices 1825.1through 1825.4. The connection engine 1800 may receive one or more inputstreams 1815 and one or more output streams 1815. While four inputdevices 1805.1 through 1805.4 and four output devices 1825.1 through1825.4 are illustrated, the subject technology may be implemented withany number of input devices and/or output devices. Specifically, thesubject technology may be implemented with one input device, two inputdevices, or more than two input devices. The subject technology may beimplemented with one output device, two output devices, or more than twooutput devices.

The input devices 1805.1 through 1805.4 may include, for example, one ormore of a keyboard, a mouse, a microphone, a camera, a touch screeninput, a universal serial bus (USB) port, etc. The input devices 1805.1through 1805.4 may reside on the connection engine 1800 or may resideremote to the connection engine 1800 and configured to communicate withthe connection engine 1800. For example, the input devices 1805.1through 1805.4 may reside on the data source 1310 of FIG. 13 or on theHTTP client 1405 of FIG. 14. The input devices 1805.1 through 1805.4 maytransmit data to the input translation block 1810.

The input translation block 1810 may be configured to receive input datafrom one or more of the input devices 1805.1 through 1805.4, totranslate the received input data to a format associated with theinput/output stream 1815, and to convert the received input data to theinput/output stream 1815.

The connection engine 1800 may be configured to receive input data fromthe input/output stream 1815, to process the input data, and to provideoutput data to the input/output stream 1815. The connection engine 1800may process the input data, for example, by making calculations with theinput data or providing the input data to the RDP client 1325 of FIGS. 1and 2 for processing. The input/output stream 1815 may also include theoutput data from the RDP client 1325.

The output translation block 1820 may be configured to receive outputdata from the input/output stream 1815, to translate the received outputdata to a format associated with the output devices 1825.1 through1825.4, and to provide the received output data to one or more of theoutput devices 1825.

The output devices 1825.1 through 1825.4 may include, for example, ascreen or display unit, an audio output unit (e.g., a speaker or aheadphone), a memory to which data may be written, etc. The outputdevices 1825.1 through 1825.4 may reside on the connection engine 1800or may reside remote to the connection engine 1800 and configured tocommunicate with the connection engine 1800. For example, the outputdevices 1825.1 through 1825.4 may reside on the data sink 1315 of FIG.13 or on the HTTP client 1405 of FIG. 14. The output devices 1825.1through 1825.4 may receive data from the output translation block 1820.

Generic Client Engine with Load Balancing

In particular embodiments, a remote computing device (e.g., remotecomputing device 1335 or HTTP client 1405) may be a “thin client.” Thethin client may include hardware (e.g., display, keyboard, mouse, etc.),and devices that may be capable of operating with a remote desktopextension protocol (e.g., RDP, ICA, VNC, PcoIP, etc.). The thin clientmay also include a web browser and may, for example, be an HTML (e.g.,HTML5) client. In particular embodiments, one or more transcodingservices or appliances (which may be implemented in hardware orvirtually in software) may interface a thin client's web browser with aremote desktop extension protocol. The transcoding service may, forexample, be launched by the GCE (e.g., as part of setup engine 1305 orconnection engine 1320). The GCE may include load balancingfunctionality or services to minimize the load on a given remote host(e.g., by determining which of multiple code instances to execute). TheGCE architecture may transcode to HTTP via websockets, HTTP tunnels, orany other suitable HTTP client compatible protocol, and this may beperformed independent of any dedicated hardware appliance or web servicethat intercepts and translates all data to/from a particular remotedesktop protocol. The GCE architecture, therefore, may enablepeer-to-peer connection and transcoding, allowing for a reduction inHTTP service overhead.

In particular embodiments, the GCE architecture includes or interactswith multiple components, to be described below. HTML syntax that isprovided by a web service to an HTML client in the form of HTML orJavaScript and which may be modified (if appropriate) by the GCE. AnHTML compatible stream may function as an interface to and from the HTMLclient using standard HTML or JavaScript compatible language forreconstructing or unpacking remote client tasks to and from the remotedesktop protocol. A remote desktop socket directed client may beincluded to transcode data to and from the HTML or remote desktopprotocol with the client-interface connection being defined by a socket(defined by a launching function). In particular embodiments, the sameclient socket may be used for all client connections using an elevatedsocket type (e.g., WS or WSS). The launching function service allows aclient to connect to a remote desktop socket directed client after loadbalancing considerations have been analyzed.

The GCE may provide a service outlined by the steps below, resulting inan HTML-transcoded, multi-session, remote desktop:

(1) The GCE may create an HTML script with connection information(defined, e.g., as a result of connection loads and policies).

(2) The HTML client may load and execute the HTML script. This may helpensure that the best host is chosen for transcoding purposes

(3) A connection page may be loaded on the HTML client. This maydescribe how the connection is to take place.

(4) The user may specify connection information including, for example,credentials. This may be required (e.g., if not hard-coded in thescript) to define connection information.

(5) The GCE awaits a connection from the HTTP compatible client. Thismay be required to receive connection arguments and provide connectionservices.

(6) The HTML client connection is assigned a unique socket by the GCE.This may be required to create a reference that may completely definethe connection information.

(7) The GCE launches a transcoding remote desktop client with the socketspecification. This may begin the transcoding operation.

(8) The HTML client terminates the connection. This may end thetranscoding operation

(9) The GCE updates its load balancing information. This step mayinclude the tallying (or accounting for) the connections, and feedingthis information back to choose the best host for transcoding purposes(e.g., in step (2)).

In this manner, a single instance of code allows for all GraphicalDevice Interface (GDI) and virtual channels to be transcoded using anHTML-compatible protocol. The transcoded client is launched with asocket defined by GCE (after analyzing load conditions on differentremote hosts). The HTML client receives (via standardHTML-method-compatible code, such as JavaScript) datastreams to use forinterpreting transcoded data. The HTML client may extract all inputstreams and apply this to localized functions and devices. The HTMLclient may encode all output streams as specified in the script(JavaScript) and applies it to remote functions and devices. Asdescribed above, GCE may provide a service to obtain user credentialsand host connection information prior to establishing a connection. GCEenables the capture of such information from a client-specified protocolto determine how a connection is to be established and may launch aunique instance to operate under those conditions. GCE may be used as astandalone service (e.g., to convert standard HTML client to theprotocol of choice, including RDP, ICA, VNC, etc. and enable fullyintegrated device I/O). GCE may also be used to enhance web-basedtranscoding services.

Generic Transcoding Service

A client (.e.g., an HTML client) may, in particular embodiments, becapable of HTML communication but not include functionality for a remotedesktop protocol (RDP). A generic transcoding service may be providedthat allows the client (e.g., an HTML5 client) to connect to an RDPserver. The generic transcoding service may allow all functions of theclient (or an application of the client) to be translated by ‘capping’the client's operation via a specified protocol. The generic transcodingservice may, for example, attach to a client (e.g., RDP client 1325) andimport and export GDI, virtual channels, and I/O devices (e.g., mouse orkeyboard) to the client. The generic transcoding service may function tofully provision typical operating system function calls and mayserialize these function calls for use by the remote client. This mayallow for end-to-end communication (including a feature-rich experience)even between incompatible protocols at the client and the server. Inparticular embodiments, the generic transcoding service may be used in astandalone manner. In yet other embodiments, the generic transcodingservice may be used in conjunction with the generic client engine (GCE)described above (either with or without load balancing). This may, forexample, provide a service that allows for any number of transcodingclient instances. The GCE, as described above, may obtain usercredentials and host connection information prior to launching aninstance of the generic transcoding client service.

In particular embodiments, the generic transcoding service may definefunctional primitives for desktop and data or device I/O import andexport, allowing for application extension for the client. The generictranscoding service may, for example, be a part of the client with apre-defined export protocol. The generic transcoding service may operatewith or include one or more of the following elements. HTML syntax maybe provided by a web service to an HTML client in the form of HTML orJavaScript. The syntax may describe all features of the extensionprotocol necessary. The syntax may also include instructions onutilizing other protocols compatible with HTML5 (including, e.g., H.264,video, audio, or other codec transport definitions). The generictranscoding service may also include an HTML-compatible stream service.The stream service provides an interface to and from the client and mayuse standard HTML or JavaScript compatible code for reconstructing orunpacking remote client tasks to and from the remote desktop protocol.The generic transcoding service may also include a remote desktop clientfunction, which may interpret, process, and transcode operations usingthe HTML-compatible stream service (e.g., rather than having devicetermination and sourcing on the client hardware).

In particular embodiments, the generic transcoding service performed forthe HTML client may include one or more of the following steps,resulting in an HTML-transcoded multi-session remote desktop:

(1) The HTML client may load and execute the HTML script, whichdescribes how the connection exchange is to take place.

(2) The user may specify connection information including, for example,credentials. The user may define connection information that may not behard-coded in the script.

(3) An HTML client connection is made through the import/export protocolservice on the transcoding remote desktop client with a particular port.This begins the transcoding operation.

(4) The HTML client terminates the connection, ending the transcodingoperation.

In particular embodiments, the generic transcoding service is used inconjunction with HTML5, such that is may translate device I/O andfunctions to and from HTML5. It may re-encode and translate HTML5-basedrenderings for client devices and functions. Client functions mayinclude, for example, display, keyboard, mouse, speaker, microphone,etc. The client functions may be terminates in a specialized exportingHTML protocol service that may be part of the RDP client (as atranscoding instance). This may permit an HTML5 client to connect to anRDP server using a newly defined HTML5 protocol. The generic transcodingservice, therefore, may be attached to the client's device functions.The HTML5 client may interface with the generic transcoding servicedirectly using websockets or HTML operations. The generic transcodingservice may split the RDP client into protocol and device pieces.

In particular embodiments, the generic transcoding service may be usedby HTTP clients by use of websockets or HTTP connections. The generictranscoding service may be used with any suitable client, including, forexample, embedded headless devices (requiring I/O but having only accessto data communications devices) or entire services providing access to atranscoding instance in the IP cloud. In particular embodiments,appending the generic transcoding service to an RDP protocol may resultin a transcoding instance whereby the protocol (RDP, ICA, or VNC) isagain encoded via a compatible client protocol directly. Device I/O atthe client may make use of this specified protocol to translateoperation of devices at the client. Applications such as desktop sharingor video streaming may, n particular embodiments, utilize the generictranscoding service to transparently export to alternate hosts forutilization. The generic transcoding service may also be used inconjunction with compression or decompression methods such as zlibcompression or decompression.

In particular embodiments, the generic transcoding service may beattached to an RDP client within a browser. For example, a full-featuredRDP instance may be created within a CHROME browser using the featuresof the Native Client environment of CHROME.

Web-Based Transcoding to Clients as a Service

In particular embodiments, a service may provide HTML clients theability to send and receive information to and from RDP servers viatranscoding protocol functions within an internet-based servicesinfrastructure (e.g., Microsoft's Internet Information Services). Thismay be done in, in one embodiment, in the following manner.

First, a browser of the HTML client (e.g., client 1405) may connect toIIS. The browser may, for example, be the CHROME browser provided byGOOGLE. The client first connects to the IIS server (e.g.,http://rdpservice.dell.com). The client may then read the index.htmlcontaining a script (e.g., JavaScript) at the IIS server. The client mayprovide connection information via a dialog box (e.g., specifying aremote RDP server). The client may then connect (e.g., by selecting a“connect” button) via the HTML client browser making a connection to ageneric client engine (e.g., part of connection engine 1320 or setupengine 1305). The client may open a connection to the GCE with argumentsindicating connection information, and the client may use anybrowser-compatible protocol including, for example, http, https, WS,WSS, or webrtc.

Once the connection between the client and the GCE is made, the GCE maycreate a new process environment and launch an RDP Client (which may,for example, include the generic transcoding service attachmentdescribed above), with the arguments specified by the user, and with thesame socket identifier as the current connection. As one example, thecommand line entry may be “rdpclient<ARGS>—icecap SOCKET—localoff.” Thismay produce a unique RDP Client instance with a connection through thespecified socket.

Next, the RDP client may make a connection as specified in the argumentsto the remote RDP Server (e.g., RDP server 1330). The HTML clientbrowser may then be used to communicate with the RDP server by use ofthe defined pathway. An equation to describe this architecture in thecontext of Microsoft Windows is the following:

HTMLc::HTMLsGTS:RDPc::RDPs:Desktop:WindowsOS:application,

where HTMLc is an HTML client, GTS stands for generic transcodingservice (Described earlier). Using this framework, any client utilizingan HTML browser may connect to an application using standard MS Windowsconvention. If it is desired to execute an application in a differentoperating system (e.g., Linux), a different protocol (e.g., VNC) may beused, described by the following equation:

HTMLc::HTMLsGTS:VNCc::VNCs:Desktop:LinuxOS:application.

In this manner, a desktop running on Linux may be exported via VNC to anHTML browser.

Runtime API Framework

In particular embodiments, a runtime API framework may be provided forremote importing and exporting of all data to or from an application.The API framework may be defined for each operating system where anapplication may be used. All remote clients may have the same userexperience wherever an application is executing using this runtime APIframework. The API framework may function without use of a remotedesktop extension protocol (e.g., RDP, ICA, VNC, etc.). The functions ofdevices may be serialized and exported or imported via the framework.For example, windowing and GDI may be exported via a defined transcodingprotocol. In this manner, the API framework may, in particularembodiments, replace (or remove the need for) certain remote desktopexporting protocols while providing all related features in a generic,multi-platform manner.

The API framework may perform multiple functions. As an example, the APIframework may include load balancing. The API framework may also includean authentication service that authenticates a user's credentials(based, e.g., on presence, time, and other factors) to providepermission to access an application. The authentication service may alsoinclude provisioning and may include services such as OAUTH2. The APIframework may also include a licensing service, which provides a licensefor a given duration or for a given purpose (based, e.g., on credentialsor other data, such as payment information). The API framework may alsoinclude an import/export service, which provides an API for attachingall import/export data stream types to/from the remote client andapplication. This import/export service may include GDI (for windowing)and remote channels such as virtual channels (for custom datatransports).

Applications may be ported or created to run atop the API framework,which itself may run on top of a given operating system. Libraries ofthe API framework may enable an application to be built and run on agiven operating system, with all I/O in its various forms imported andexported to a remote client. If the framework is used with the GCE(described above), the framework may allow for concurrent,multi-session, multi-instance operation of the same application fordifferent users. The framework may, however, function without use of theGCE.

The framework may, therefore, provide a means foroperating-system-agnostic operations that provide authentication,licensing, exporting of GDI, importing/exporting of video and audio, andgeneral virtual channel data to an application running on any operatingsystem. This may, therefore, prevent the need for a complex ortime-consuming port of an operating-system-specific code base.

In particular embodiments, the API framework may operate in conjunctionwith the generic transcoding service (described earlier) to render theconventions of an operating system irrelevant for use of an application.The following equation expresses the operation:

HTMLc::HTMLsGTS:APIFrameWork:OS:application

This equation embodies an alternative to a special desktop exportingprotocol. The framework exports the entire desktop, and all logicaldevice functions are a function of the framework. The API framework neednot be contained with an operating system where the application ofinterest is executed. Rather, the framework may function atop anexisting operating system and, for example, validate user credentialsand licenses where applicable. The API framework in combination with thegeneric transcoding service may be located on the same host or on anyhost having authorization authority. Any HTTP browser may, therefore,connect to or open any application on any host after a server validatescredentials, license, and payment information by using the APIframework. FIG. 20 illustrates an example of the API framework operatingin conjunction with the GCE. In this example, any HTTP web browser mayopen any application on any host after the server validates credentials,licensing, and payment information.

In particular embodiments, the API framework in combination with thegeneric transcoding service may use a functional API to intercept systemcalls for I/O and related functions. A functional API may be, forexample, IPC, socket, function interface, etc. The functional API mayreceive data related to a standard localized function and export thisdata. The process of converting data designed for a local function andexporting for use or operation on a different device is referred to as“serialization”. Device-related functions may be serialized through acommunication method. Such a method may “serialize” arguments andfunction descriptions. For instance, Graphical Device Interface (GDI)may require the x,y coordinate and data representing pixel informationfor proper operation. Moreover, additional information may define bitdepth (number of bits used to describe a pixel). Such an examplefunction would be called as follows:

code1: write_Gdi(x1, y1, x2, y2, bits, Pixel_data);

This may be serialized by encoding through a communications pathway thesame information as follows:

code2: send(DESTINATION, “write_gdi”, x1, y1, x2, y2, bits, Pixel_data);

code2 may be the same information encoded using a transport. The name ofthe function may also be encoded as literal text. The function may nowbe exported to a remote host, where the operation may take place asfollows:

code3: packet=receive(SOURCE);

The entire operation may now be placed into a packet string on thereceiving host. This may be further parsed for data as follows:

code4: operation=GET_OPERATION(packet); //this will retrieve theoperation to perform (write_gdi)

Once the operation is received, arguments related to this operation maybe satisfied:

code5: if(operation == “write_gdi”) { x1-coord = GET_ARGUMENT(packet);y1-coord = GET_ARGUMENT(packet); ... }

Finally, the operation may be executed on the DESTINATION HOST:

code6: write_Gdi(x1, y1, x2, y2, bits, Pixel_data);

In one aspect, any methods, instructions, code, means, logic,components, blocks, modules and the like (e.g., software or hardware)described or claimed herein can be represented in drawings (e.g., flowcharts, block diagrams), such drawings (regardless of whether explicitlyshown or not) are expressly incorporated herein by reference, and suchdrawings (if not yet explicitly shown) can be added to the disclosurewithout constituting new matter. For brevity, some (but not necessarilyall) of the clauses/descriptions/claims are explicitly represented indrawings, but any of the clauses/descriptions/claims can be representedin drawings in a manner similar to those drawings explicitly shown. Forexample, a flow chart can be drawn for any of the clauses, sentences orclaims for a method such that each operation or step is connected to thenext operation or step by an arrow. In another example, a block diagramcan be drawn for any of the clauses, sentences or claims havingmeans-for elements (e.g., means for performing an action) such that eachmeans-for element can be represented as a module for element (e.g., amodule for performing an action).

Those of skill in the art would appreciate that items such as thevarious illustrative blocks, modules, elements, components, methods,operations, steps, and algorithms described herein (e.g., the serviceincompatible client 210, the proxy machine 220, the firewall 120, thebinding interface 130, the network-based procedure call interface 140,the gateway interface 150, the remote server computing device 160, andthe components therein) may be implemented as hardware, computersoftware, or a combination of both.

To illustrate the interchangeability of hardware and software, itemssuch as the various illustrative blocks, modules, elements, components,methods, operations, steps, and algorithms have been described generallyin terms of their functionality. Whether such functionality isimplemented as hardware or software depends upon the particularapplication and design constraints imposed on the overall system.Skilled artisans may implement the described functionality in varyingways for each particular application.

In one aspect, “means,” a block, a module, an element, a component or aprocessor may be an item (e.g., one or more of blocks, modules,elements, components or processors) for performing one or more functionsor operations. In one aspect, such an item may be an apparatus,hardware, or a portion thereof. In one example, an item may have astructure in the form of, for example, an instruction(s) for performingthe function(s) or operation(s), where the instruction(s) are encoded orstored on a machine-readable medium, on another device, or on a portionthereof, where an instruction(s) may be software, an application(s), asubroutine(s), or a portion thereof. In an example, an item may beimplemented as one or more circuits configured to perform thefunction(s) or operation(s). A circuit may include one or more circuitsand/or logic. A circuit may be analog and/or digital. A circuit may beelectrical and/or optical. A circuit may include transistors. In anexample, one or more items may be implemented as a processing system(e.g., a digital signal processor (DSP), an application specificintegrated circuit (ASIC), a field programmable gate array (FPGA), etc.,or a portion or a combination of any of the foregoing). Those skilled inthe art will recognize how to implement the instructions, circuits, andprocessing systems.

A reference to an element in the singular is not intended to mean “oneand only one” unless specifically so stated, but rather “one or more.”For example, a message may refer to one or more messages.

Unless specifically stated otherwise, the term “some” refers to one ormore. Pronouns in the masculine (e.g., his) include the feminine andneuter gender (e.g., her and its) and vice versa. Headings andsubheadings, if any, are used for convenience only and do not limit theinvention.

The word “exemplary” is used herein to mean “serving as an example orillustration.” Any aspect or design described herein as “exemplary” isnot necessarily to be construed as preferred or advantageous over otheraspects or designs. In one aspect, various alternative configurationsand operations described herein may be considered to be at leastequivalent.

A phrase such as an “aspect” does not imply that such aspect isessential to the subject technology or that such aspect applies to allconfigurations of the subject technology. A disclosure relating to anaspect may apply to all configurations, or one or more configurations.An aspect may provide one or more examples. A phrase such as an aspectmay refer to one or more aspects and vice versa. A phrase such as an“embodiment” does not imply that such embodiment is essential to thesubject technology or that such embodiment applies to all configurationsof the subject technology. A disclosure relating to an embodiment mayapply to all embodiments, or one or more embodiments. An embodiment mayprovide one or more examples. A phrase such an embodiment may refer toone or more embodiments and vice versa. A phrase such as a“configuration” does not imply that such configuration is essential tothe subject technology or that such configuration applies to allconfigurations of the subject technology. A disclosure relating to aconfiguration may apply to all configurations, or one or moreconfigurations. A configuration may provide one or more examples. Aphrase such a configuration may refer to one or more configurations andvice versa.

In one aspect of the disclosure, when actions or functions are describedas being performed by an item (e.g., receiving, determining, providing,generating, converting, displaying, notifying, accepting, selecting,controlling, transmitting, reporting, sending, authenticating,verifying, binding, creating, or any other action or function), it isunderstood that such actions or functions may be performed by the itemdirectly. In another example, when an item is described as performing anaction, the item may be understood to perform the action indirectly, forexample, by facilitating (e.g., enabling, causing or performing aportion of) such an action. For example, generating can refer tofacilitating generation. In one aspect, performing an action may referto performing a portion of the action (e.g., performing a beginning partof the action, performing an end part of the action, or performing amiddle portion of the action).

In one aspect, unless otherwise stated, all measurements, values,ratings, positions, magnitudes, sizes, and other specifications that areset forth in this specification, including in the claims that follow,are approximate, not exact. In one aspect, they are intended to have areasonable range that is consistent with the functions to which theyrelate and with what is customary in the art to which they pertain.

In one aspect, the term “coupled” or the like may refer to beingdirectly coupled. In another aspect, the term “coupled” or the like mayrefer to being indirectly coupled.

Various items may be arranged differently (e.g., arranged in a differentorder, or partitioned in a different way) all without departing from thescope of the subject technology. In one aspect of the disclosure, theelements recited in the accompanying claims may be performed by one ormore modules or sub-modules.

It is understood that the specific order or hierarchy of steps,operations or processes disclosed is an illustration of exemplaryapproaches. Based upon design preferences, it is understood that thespecific order or hierarchy of steps, operations or processes may berearranged. Some of the steps, operations or processes may be performedsimultaneously. Some or all of the steps, operations, or processes maybe performed automatically, without the intervention of a user. Theaccompanying method claims, if any, present elements of the varioussteps, operations or processes in a sample order, and are not meant tobe limited to the specific order or hierarchy presented.

The disclosure is provided to enable any person skilled in the art topractice the various aspects described herein. The disclosure providesvarious examples of the subject technology, and the subject technologyis not limited to these examples. Various modifications to these aspectswill be readily apparent to those skilled in the art, and the genericprinciples defined herein may be applied to other aspects.

All structural and functional equivalents to the elements of the variousaspects described throughout this disclosure that are known or latercome to be known to those of ordinary skill in the art are expresslyincorporated herein by reference and are intended to be encompassed bythe claims. Moreover, nothing disclosed herein is intended to bededicated to the public regardless of whether such disclosure isexplicitly recited in the claims. No claim element is to be construedunder the provisions of 35 U.S.C. §112, sixth paragraph, unless theelement is expressly recited using the phrase “means for” or, in thecase of a method claim, the element is recited using the phrase “stepfor.” Furthermore, to the extent that the term “include,” “have,” or thelike is used, such term is intended to be inclusive in a manner similarto the term “comprise” as “comprise” is interpreted when employed as atransitional word in a claim.

The Title, Background, Summary, Brief Description of the Drawings andAbstract of the disclosure are hereby incorporated into the disclosureand are provided as illustrative examples of the disclosure, not asrestrictive descriptions. It is submitted with the understanding thatthey will not be used to limit the scope or meaning of the claims. Inaddition, in the Detailed Description, it can be seen that thedescription provides illustrative examples and the various features aregrouped together in various embodiments for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting an intention that the claimed subject matter requires morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter lies in less than allfeatures of a single disclosed configuration or operation. The followingclaims are hereby incorporated into the Detailed Description, with eachclaim standing on its own as a separately claimed subject matter.

The claims are not intended to be limited to the aspects describedherein, but is to be accorded the full scope consistent with thelanguage claims and to encompass all legal equivalents. Notwithstanding,none of the claims are intended to embrace subject matter that fails tosatisfy the requirement of 35 U.S.C. §101, 102, or 103, nor should theybe interpreted in such a way. Any unintended embracement of such subjectmatter is hereby disclaimed.

1. A method comprising, by a computing device: receiving device datafrom a client device; encoding the data for transmission to a remotedesktop client, the re-encoding being performed according to apre-defined import/export protocol.
 2. The method of claim 1, whereinthe device data comprises input/output data from the client device. 3.The method of claim 1, wherein the client device comprises one or moreof: a mouse; a touch device; a keyboard; a display; a speaker; or amicrophone.
 4. The method of claim 1, wherein the import/export protocolis HTML5-compatible.
 5. The method of claim 1, wherein the datacomprises a rendering associated with the client device.
 6. The methodof claim 1, wherein the receiving and encoding are performed by a remotedesktop protocol client.
 7. The method of claim 6, wherein the remotedesktop protocol client is launched by a general client engine.
 8. Oneor more computer-readable non-transitory storage media embodyingsoftware that is operable when executed to: receive device data from aclient device; encode the data for transmission to a remote desktopclient, the re-encoding being performed according to a pre-definedimport/export protocol.
 9. The media of claim 8, wherein the device datacomprises input/output data from the client device.
 10. The media ofclaim 8, wherein the client device comprises one or more of: a mouse; atouch device; a keyboard; a display; a speaker; or a microphone.
 11. Themedia of claim 8, wherein the import/export protocol isHTML5-compatible.
 12. The media of claim 8, wherein the data comprises arendering associated with the client device.
 13. The media of claim 8,wherein the receiving and encoding are performed by a remote desktopprotocol client.
 14. The media of claim 13, wherein the remote desktopprotocol client is launched by a general client engine.
 15. A systemcomprising: a computing device comprising one or more processors; and amemory coupled to the processors comprising instructions executable bythe processors, the processors operable when executing the instructionsto: receive device data from a client device; encode the data fortransmission to a remote desktop client, the re-encoding being performedaccording to a pre-defined import/export protocol.
 16. The system ofclaim 15, wherein the device data comprises input/output data from theclient device.
 17. The system of claim 15, wherein the client devicecomprises one or more of: a mouse; a touch device; a keyboard; adisplay; a speaker; or a microphone.
 18. The system of claim 15, whereinthe import/export protocol is HTML5-compatible.
 19. The system of claim15, wherein the data comprises a rendering associated with the clientdevice.
 20. The system of claim 15, wherein the receiving and encodingare performed by a remote desktop protocol client.